It feels like this sort of thing is better than nothing, but I wish we had something better.

The existing epoch paper is pretty good, but doesn't directly target LLMs in a way which seems somewhat sad.

The thing I'd be most excited about is:

• Epoch does an in depth investigation using an estimation methodology which is directly targeting LLMs (rather than looking at returns in some other domains).
• They use public data and solicit data from companies about algorithmic improvement, head count, compute on experiments etc.
• (Some) companies provide this data. Epoch potentially doesn't publish this exact data and instead just publishes the results of the final analysis to reduce capabilities externalities. (IMO, companies are somewhat unlikely to do this, but I'd like to be proven wrong!)

I think usefully massively scaling up funding and the field is hard in various ways, especially doing so very quickly, but I roughly agree if we accept that premise. Overall, I think this is only a small quantitative effect in short timelines because it doesn't seem that likely, even if it happens it seems likely to not be that good given various difficulties in scaling, and even if it is good, I think the chance of huge improvements isn't that high (given my understanding of where we are at in the current returns curve). TBC, I think a well run massive AI safety program would greatly lower risk in expectation.

I think you are correct with respect to my estimate of $\alpha$ and the associated model I was using. Sorry about my error here. I think I was fundamentally confusing a few things in my head when writing out the comment.

I think your refactoring of my strategy is correct and I tried to check it myself, though I don't feel confident in verifying it is correct.

Your estimate doesn't account for the conversion between algorithmic improvement and labor efficiency, but it is easy to add this in by just changing the historical algorithmic efficiency improvement of 3.5x/year to instead be the adjusted effective labor efficiency rate and then solving identically. I was previously thinking the relationship was that labor efficiency was around the same as algorithmic efficiency, but I now think this is more likely to be around $algo\_efficienc{y}^2$ based on Tom's comment.

Plugging this is, we'd get:

$$\frac{\lambda }{\beta }(1-p)=\frac{r}{q}(1-p)=\frac{\ln \left({3.5}^2\right)}{0.4\ln \left(4\right)+0.6\ln \left(1.6\right)}(1-0.4)=2\frac{\ln \left(3.5\right)}{\ln \left(2.3\right)}(1-0.4)=2\cdot 1.5\cdot 0.6=1.8$$

(In your comment you said $\frac{\ln \left(3.5\right)}{\ln \left(2.3\right)}=1.6$, but I think the arithmetic is a bit off here and the answer is closer to 1.5.)

(I'm going through this and understanding where I made an error with my approach to $\alpha$. I think I did make an error, but I'm trying to make sure I'm not still confused.)

Wikipedia says Cobb-Douglas functions can have the exponents not add to 1 (while both being between 0 and 1). Maybe this makes sense here? Not an expert.

It shouldn't matter in this case because we're raising the whole value of $E$ to $\lambda$.

I think Tom's take is that he expects I will put more probability on software only singularity after updating on these considerations. It seems hard to isolate where Tom and I disagree based on this comment, but maybe it is on how much to weigh various considerations about compute being a key input.

My sense is that I start with a higher $r$ value due to the LLM case looking faster (and not feeling the need to adjust downward in a few places like you do in the LLM case). Obviously the numbers in the LLM case are much less certain given that I'm guessing based on qualitative improvement and looking at some open source models, but being closer to what we actually care about maybe overwhelms this.

I also think I'd get a slightly lower update on the diminishing returns case due to thinking it has a good chance of having substantially sharper dimishing returns as you get closer and closer rather than having linearly decreasing $r$ (based on some first principles reasoning and my understanding of how returns diminished in the semi-conductor case).

But the biggest delta is that I think I wasn't pricing in the importance of increasing capabilities. (Which seems especially important if you apply a large R&D parallelization penalty.)

Yep, I think my estimates were too low based on these considerations and I've updated up accordingly. I updated down on your argument that maybe $r$ decreases linearly as you approach optimal efficiency. (I think it probably doesn't decrease linearly and instead drops faster towards the end based partially on thinking a bit about the dynamics and drawing on the example of what we've seen in semi-conductor improvement over time, but I'm not that confident.) Maybe I'm now at like 60% software-only is feasible given these arguments.

I'm citing the polls from Daniel + what I've heard from random people + my guesses.

Interesting. My numbers aren't very principled and I could imagine thinking capability improvements are a big deal for the bottom line.

Yeah, I'm trying to include delay as fine.

I'm just trying to point at "the point when aggressive intervention by a bunch of parties is potentially still too late".