Summary: One area we plan to explore at Resolution is personas and character training, operationalized as finding and controlling low-dimensional structure in models that emerges in pretraining and flows through post-training to superintelligence. The hope is to expand and systematize phenomena such as emergent misalignment, subliminal learning, and other empirical persona research, then intervene on this structure without accidentally hiding undesirable behavior elsewhere. If this approach resonates with you, consider working with us.

Glimmers of low-dimensional structure

Our understanding of AI training and alignment as a field is very poor. If sufficient alignment of superintelligent AI agents requires pinning down the precise meaning of alignment and turning that meaning into high-accuracy training data and algorithms, we are likely to fail. Modern LLMs have trillions of parameters: our understanding...

For the past year, we at the AI Futures Project have been sinking most of our time into our next big scenario. Now it’s done! 

It’s called AI 2040: Plan A.

It’s called Plan A because it’s a recommendation, not a prediction. It’s what we think should happen, not what will happen, though we think it’s plausible enough to aim for.

It’s called AI 2040 because in it, they delay the creation of superintelligence to 2040. It would have happened much sooner (in 2030, to be precise) if not for decisive action on the part of the US and Chinese governments. 

As with AI 2027, summaries don’t really do it justice, since the whole point was to be detailed and comprehensive and work things out step by step rather than rely on high-level abstractions like doom or utopia. 

Read the scenario at ai-2040.com. You can...

Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:

• Defying prestigious dogma and searching for raw truth;
• Defying social pressure, acting alone to help someone while others watch;
• Defying your self-expectations (your “role”), instead searching over lines of cause-and-effect to find a winning pathway;
• Defying a powerful foe’s threats, because they only threaten since people like you cave;
• Defying the specter of apparent impossibility because you can’t bear to lose.

I cannot return to you and say “I defied and then I won.” But I’m at least here to say “I defied.”

I recommend reading this article on my website since the embeds and typography work better there: click here.

Why I left Google DeepMind

In January, Department of Homeland Security (DHS) officers...

When we get explicit strong generalisation to work (see the first post on the matter and the second) my dream would be to create pre-aligned generalising AIs.

Think about the usual conflict between alignment and capabilities, between doing the right thing and doing the easy thing. The standard narrative puts the good people at a constant disadvantage: they have to carefully plan every AI advance, always on the lookout for potential dangers. While those who don’t care can just YOLO and let it rip and let their AIs get ever more powerful without taking any responsibility.

Now, in reality, there is some nuance to the story; but I don’t want to nuance it, I want to turn it on its head. I want to create AIs so that the...

We hand coded weights for one layer MLPs that memorises labels for input token sequences of length two. The number of facts our hand-coded models can memorise with 90% accuracy^[1]scales roughly linearly with the models' parameter count^[2], just like trained models for the same architecture. However, our hand-coded models' scaling prefactor still falls short of trained models' by a factor of . A hybrid solution in which we handcode the MLP input weights and learn the MLP output weights, which amounts to a linear classification problem^[3], falls short by a factor of .

We pose a challenge to the community: Find better...