Regarding direction 17: There might be some potential drawbacks to ADAM. I think its possible that some very agentic programs have relatively low $g$ score. This is due to explicit optimization algorithms being low complexity.

(Disclaimer: the following argument is not a proof, and appeals to some heuristics/etc. We fix $M=M_0$ for these considerations too.) Consider an utility function $\hat{U}$. Further, consider a computable approximation of the optimal policy (AIXI that explicitly optimizes for $\hat{U}$) and has an approximation parameter n (this could be AIXI-tl, plus some approximation of $\hat{U}$; higher $n$ is better approximation). We will call this approximation of the optimal policy ${\pi }_n^{\hat{U}}$. This approximation algorithm has complexity $K\left({\pi }_n^{\hat{U}}\right)=C+K\left(\hat{U}\right)+K\left(n\right)$, where $C>0$ is a constant needed to describe the general algorithm (this should not be too large).

We can get better approximation by using a quickly growing function, such as the Ackermann function with $n=A(k,k)$. Then we have $K\left({\pi }_{A(k,k)}^{\hat{U}}\right)=C+K\left(\hat{U}\right)+K\left(A\right(k,k\left)\right)\le C+K\left(\hat{U}\right)+\log \left(k\right)$.

What is the $g$ score of this policy? We have $g\left({\pi }_{A(k,k)}^{\hat{U}}\right)={max}_U\left({min}_{{\pi }^{\prime }:\dots }K\right({\pi }^{\prime })-K(U\left)\right)$. Let $\overline{U}$ be maximal in this expression. If $K\left(\overline{U}\right)\ge K\left(\hat{U}\right)-C$, then $$g\left({\pi }_{A(k,k)}^{\hat{U}}\right)=\underset{{\pi }^{\prime }:E_{{\zeta }_{M_0}{\pi }^{\prime }}\left(\overline{U}\right)\ge E_{{\zeta }_{M_0}{\pi }_{A(k,k)}^{\hat{U}}}\left(\overline{U}\right)}{min}K\left({\pi }^{\prime }\right)-K\left(\overline{U}\right)\le K\left({\pi }_{A(k,k)}^{\hat{U}}\right)-K\left(\hat{U}\right)+C\le 2C\log \left(k\right)$$.

For the other case, let us assume that if $K\left(\overline{U}\right)<K\left(\hat{U}\right)-C$, the policy ${\pi }_{A(k,k)}^{\overline{U}}$ is at least as good at maximizing $\overline{U}$ than ${\pi }_{A(k,k)}^{\hat{U})}$. Then, we have $$g\left({\pi }_{A(k,k)}^{\hat{U}}\right)=\underset{{\pi }^{\prime }:E_{{\zeta }_{M_0}{\pi }^{\prime }}\left(\overline{U}\right)\ge E_{{\zeta }_{M_0}{\pi }_{A(k,k)}^{\hat{U}}}\left(\overline{U}\right)}{min}K\left({\pi }^{\prime }\right)-K\left(\overline{U}\right)\le K\left({\pi }_{A(k,k)}^{\overline{U}}\right)-K\left(\overline{U}\right))\le C+\log (k)$$.

I don't think that the assumption ($({\pi }_{A(k,k)}^{\overline{U}}$ maximizes $barU$ better than $({\pi }_{A(k,k)}^{\hat{U}}$) is true for all $\hat{U}$ and $k$, but plausibly we can select $\hat{U}$ such that this is the case (exceptions, if they exist, would be a bit weird, and if ADAM working well due to these weird exceptions feels a bit disappointing to me). A thing that is not captured by approximations such as AIXI-tl are programs that halt but have insane runtime (longer than $A(k,k)$). Again, it would feel weird to me if ADAM sort of works because of low-complexity extremely-long-running halting programs.

To summarize, maybe there exist policies ${\pi }_{A(k,k)}^{\hat{U}}$ which strongly optimize a non-trivial utility function $\hat{U}$ with approximation parameter $n=A(k,k)$, but where $g\left({\pi }_{A(k,k)}^{\hat{U}}\right)\le 2C+\log \left(k\right)$ is relatively small.

I am going to assume that in the code, when calculating p_alice_win_given_not_caught, we do not divide the ${\lambda }_2$ term by two (since this is not that consistent with the description. I am also assuming that $0.25-t_2$ is a typo and $0.5-t_2$ is meant, which would also be more consistent with other stuff). So I am going to assume assume a symmetrical version.

Here, P(Alice wins) is $e^{-{\lambda }_1{t}_1-{\lambda }_2{t}_2}-e^{-{\lambda }_1/2-{\lambda }_2/2}=:f({\lambda }_1,{\lambda }_2,t_1,t_2)$. Wlog we can assume $0<t<1$ (otherwise Bob will run everything or nothing in shielded mode).

We claim that $\left(\right({\lambda }_1^{\ast },{\lambda }_2^{\ast }),(t_1^{\ast },t_2^{\ast }\left)\right):=\left(\right({\mu }^{\ast },{\mu }^{\ast }),(t/2,t/2)$ is a (pure) Nash equilibrium, where ${\mu }^{\ast }=\log t/(t-1)$.

To verify, lets first show that Alice cannot make a better choice if Bob plays $(t_1^{\ast },t_2^{\ast })=(t/2,t/2)$. We have $f({\lambda }_1,{\lambda }_2,t_1^{\ast },t_2^{\ast })=e^{-({\lambda }_1+{\lambda }_2)t/2}-e^{-({\lambda }_1+{\lambda }_2)/2}$. Since this only depends on the sum, we can make the substitution $\mu =({\lambda }_1+{\lambda }_2)/2$. Thus, we want to maximize $\tilde{f}(\mu ,t)=e^{-\mu t}-e^{-\mu }$. We have ${\tilde{f}}^{\prime }(\mu ,t)=-t{e}^{-\mu t}+e^{-\mu }=0$. Rearranging, we get $t{e}^{-\mu t}=e^{-\mu }$. Taking logs, we get $\log t-\mu t=-\mu$. Rearranging, we get $\mu =\log t/(t-1)$. Thus, $\mu ={\mu }^{\ast }$ is the optimal choice. This means, that if Bob sticks to his strategy, Alice cannot do better than $f({\mu }^{\ast },{\mu }^{\ast },t/2,t/2)$.

Now, lets show that Bob cannot do better. We have $f({\lambda }_1^{\ast },{\lambda }_2^{\ast },t_1,t_2)=e^{-{\mu }^{\ast }{t}_1-{\mu }^{\ast }{t}_2}-e^{-{\mu }^{\ast }/2-{\mu }^{\ast }/2}=e^{-{\mu }^{\ast }t}-e^{-{\mu }^{\ast }}$. This does not depend on $t_1$ and $t_2$ anymore, so any choice of $t_1$ and $t_2$ is optimal if Alice plays $({\mu }^{\ast },{\mu }^{\ast })$.

(If I picked the wrong version of the question, and you actually want some symmetry: I suspect that the solution will have similarities, or that in some cases the solution can be obtained by rescaling the problem back into a more symmetric form.)

Nanotech industry-rebuilding comes earlier than von Neumann level? I doubt that. A lot of existing people are close to von Neumann level.

Maybe your argument is that there will be so many AGIs, that they can do Nanotech industry rebuilding while individually being very dumb. But I would then argue that the collective already exceeds von Neumann or large groups of humans in intelligence.