Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:
I cannot return to you and say “I defied and then I won.” But I’m at least here to say “I defied.”
I recommend reading this article on my website since the embeds and typography work better there: click here.
In January, Department of Homeland Security (DHS) officers...
Also available in markdown at theMultiplicity.ai/blog/schelling-goodness.
This post explores a notion I'll call Schelling goodness. Claims of Schelling goodness are not first-order moral verdicts like "X is good" or "X is bad." They are claims about a class of hypothetical coordination games in the sense of Thomas Schelling, where the task being coordinated on is a moral verdict. In each such game, participants aim to give the same response regarding a moral question, by reasoning about what a very diverse population of intelligent beings would converge on, using only broadly shared constraints: common knowledge of the question at hand, and background knowledge from the survival and growth pressures that shape successful civilizations. Unlike many Schelling coordination games, we'll be focused on scenarios with no shared history or knowledge...
Another rough alignment idea.
The non-overlap principle. The basic idea, identify the convergent instrumentally useful cognition, and avoid it.
Suppose you ask the AI to cure cancer, and to find a room temperature superconductor. This principle says that there should be 0 overlap between the thought processes used in these 2 ideas. The cancer curing AI should avoid doing any thinking that would be useful in general. Figuring out how to hack it's way out and access the internet is useful in general. Sure, there are things that are generally useful that migh... (read more)