Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:
I cannot return to you and say “I defied and then I won.” But I’m at least here to say “I defied.”
I recommend reading this article on my website since the embeds and typography work better there: click here.
In January, Department of Homeland Security (DHS) officers...
Also available in markdown at theMultiplicity.ai/blog/schelling-goodness.
This post explores a notion I'll call Schelling goodness. Claims of Schelling goodness are not first-order moral verdicts like "X is good" or "X is bad." They are claims about a class of hypothetical coordination games in the sense of Thomas Schelling, where the task being coordinated on is a moral verdict. In each such game, participants aim to give the same response regarding a moral question, by reasoning about what a very diverse population of intelligent beings would converge on, using only broadly shared constraints: common knowledge of the question at hand, and background knowledge from the survival and growth pressures that shape successful civilizations. Unlike many Schelling coordination games, we'll be focused on scenarios with no shared history or knowledge...
Imagine you have a small corpus of data from good ethical people, and a large corpus of data that's full of 4chan trolls or reward hacking or some other bad thing. (The large corpus isn't just trolls, there is useful data in there too)
The hoped for benefit is that the AI displays all the useful knowledge that can be extracted from the large corpus, but only displays the behavior patterns found in the small corpus.
So suppose the 4chan data contains a few useful maths ideas, and also a lot of slurs. And suppose the good data consists entirely of maths proof... (read more)