This is a linkpost for https://www.ai-2040.com/
Total research transparency feels a bit too galaxy-brained for me. It makes non-robust assumptions that newly discovered techniques won't be usable to enhance already existing open-weights models to excessively dangerous capability levels. I also think the disincentive for research is overstated as it neglects first-mover advantage.
I'm excited about people commenting on this post with questions, feedback, critiques, different proposals, etc. We'll try to monitor it and respond to many of the comments.
A key part of this strategy is deterrence, "Mutually Assured Compute Destruction" which gets its own section. It doesn't mention the generalization Mutually Assured AI Malfunction (MAIM) from Schmidt, Wang, and me last year. This also spends 1/3 of the MAIM discussion on verification and how to do this in a multilateral way.
Meanwhile it cites other works like A Narrow Path. I even left this feedback to you all at AI Futures before this was released. This would constitute plagiarism in any other context. It's a bewildering unforced error--it's extremely related, it's a certainly a nontrivial idea, and I told you all this recently--I hope you all fix it.
For the past year, we at the AI Futures Project have been sinking most of our time into our next big scenario. Now it’s done!
It’s called AI 2040: Plan A.
It’s called Plan A because it’s a recommendation, not a prediction. It’s what we think should happen, not what will happen, though we think it’s plausible enough to aim for.
It’s called AI 2040 because in it, they delay the creation of superintelligence to 2040. It would have happened much sooner (in 2030, to be precise) if not for decisive action on the part of the US and Chinese governments.
As with AI 2027, summaries don’t really do it justice, since the whole point was to be detailed and comprehensive and work things out step by step rather than rely on high-level abstractions like doom or utopia.
Read the scenario at ai-2040.com. You can listen to it on audio, or view it on mobile, but the experience is significantly better on a normal computer.
What’s next for us?
Well, first we are going to respond to comments and otherwise engage with whatever conversation, responses, critiques, etc. that AI 2040: Plan A sparks. Beyond that, we aren’t sure yet. In general our mission is to help make AGI go well, and now we’ve tried out both forecasting and planning. Maybe we’ll get started on another big scenario. On the other hand, these megaprojects take so much time…