Preface for LessWrong: When I think back on my most cherished memories of this community, I return to those honoring defiance in pursuit of goodness:
I cannot return to you and say “I defied and then I won.” But I’m at least here to say “I defied.”
I recommend reading this article on my website since the embeds and typography work better there: click here.
In January, Department of Homeland Security (DHS) officers...
Also available in markdown at theMultiplicity.ai/blog/schelling-goodness.
This post explores a notion I'll call Schelling goodness. Claims of Schelling goodness are not first-order moral verdicts like "X is good" or "X is bad." They are claims about a class of hypothetical coordination games in the sense of Thomas Schelling, where the task being coordinated on is a moral verdict. In each such game, participants aim to give the same response regarding a moral question, by reasoning about what a very diverse population of intelligent beings would converge on, using only broadly shared constraints: common knowledge of the question at hand, and background knowledge from the survival and growth pressures that shape successful civilizations. Unlike many Schelling coordination games, we'll be focused on scenarios with no shared history or knowledge...
The anti-Newcomb problem is a puzzle proposed by LessWrong user @drnickbone as a challenge to UDT. I rediscovered it in 2015 and gave it a name. Recently, Richard Ngo mentioned a reminiscent but actually pretty different puzzle called "Newcomb's revenge". All of those are cases in which CDT apparently outperforms FDT.
Of course, one can always construct a decision problem that diagonalizes against any given decision theory. The interesting question is defining a natural class of "fair" problems for which a given decision theory is optimal. One can then argu... (read more)