AI ALIGNMENT FORUM
AF

All of DirectedEvolution's Comments + Replies

My Objections to "We’re All Gonna Die with Eliezer Yudkowsky"
DirectedEvolution1937

If that were the case, I actually would fault Eliezer, at least a little. He’s frequently, though by no means always, stuck to qualitative and hard-to-pin-down punditry like we see here, rather than to unambiguous forecasting.

This allows him, or his defenders, to retroactively defend his predictions as somehow correct even when they seem wrong in hindsight.

Let’s imagine for a moment that Eliezer’s right that AI safety is a cosmically important issue, and yet that he’s quite mistaken about all the technical details of how AGI will arise and how to effective... (read more)

Reply
1Alex Turner
I think several things here, considering the broader thread:  1. You've done a great job in communicating several reactions I also had: 1. There are signs of serious mispredictions and mistakes in some of the 2008 posts. 2. There are ways to read these posts as not that bad in hindsight, but we should be careful in giving too much benefit of the doubt. 3. Overall these observations constitute important evidence on EY's alignment intuitions and ability to make qualitative AI predictions. 2. I did a bad job of marking my interpretations of what Eliezer wrote, as opposed to claiming he did dismiss ANNs. Hopefully my edits have fixed my mistakes.
My Objections to "We’re All Gonna Die with Eliezer Yudkowsky"
DirectedEvolution1044

One of Yudkowsky's claims in the post you link is:

It's hard to build a flying machine if the only thing you understand about flight is that somehow birds magically fly.  What you need is a concept of aerodynamic lift, so that you can see how something can fly even if it isn't exactly like a bird.

This is a claim that lack of the correct mechanistic theory is a formidable barrier for capabilities, not just alignment, and it inaccurately underestimates the amount of empirical understandings available on which to base an empirical approach.

It's true that ... (read more)

Reply
4Alex Turner
To be fair, he said that those two will work, and (perhaps?) admitted the possibility of "run advanced neural network algorithms" eventually working. Emphasis mine:
0rvnnt
I think it might be relevant to note here that it's not really humans who are building current SOTA AIs --- rather, it's some optimizer like SGD that's doing most of the work. SGD does not have any mechanistic understanding of intelligence (nor anything else). And indeed, it takes a heck of a lot of data and compute for SGD to build those AIs. This seems to be in line with Yudkowsky's claim that it's hard/inefficient to build something without understanding it. I think it's important to distinguish between * Scaling up a neural network, and running some kind of fixed algorithm on it. * Scaling up a neural network, and using SGD to optimize the parameters of the NN, so that the NN ends up learning a whole new set of algorithms. IIUC, in Artificial Mysterious Intelligence, Yudkowsky seemed to be saying that the former would probably fail. OTOH, I don't know what kinds of NN algorithms were popular back in 2008, or exactly what NN algorithms Yudkowsky was referring to, so... *shrugs*.