A putative new idea for AI control; index here.

A simple and easy design for a $u$-maximising agent that turns into a $u$-minimising one.

Let $X$ be some boolean random variable outside the agent's control, that will be determined at some future time $t$ (based on a cosmic event, maybe?). Set it up so that $P(X=1)=ϵ$, and for a given utility $u$ consider the utility:

• $u^{\#}=\left(2/ϵ\right)Xu-u$.

Before $t$, the expected value of $\left(2/ϵ\right)X$ is $2$, so $u^{\#}=u$. Hence the agent is a $u$-maximiser. After $t$, the most likely option is $X=0$, hence a little bit of evidence to that effect is enough to make $u^{\#}$ into a $u$-minimiser.

This isn't perfect corrigibility --- the agent would be willing to sacrifice a bit of $u$-value (before $t$) in order to maintain its flexibility after $t$. To combat this effect, we could instead use:

• $u^{\#}=\Omega \left(2/ϵ\right)Xu-u$.

If $\Omega$ is large, then the agent is willing to pay very little $u$-value to maintain flexibility. However, the amount of evidence of $X=0$ that it needs to become a $u$-minimiser is equally proportional to $\Omega$, so $X$ better be a clear and convincing event.