Cortés, Pizarro, and Afonso as Precedents for Takeover
Cortés, Pizarro, and Afonso as Precedents for Takeover
Very interesting post! However, I have a big disagreement with your interpretation of why the European conquerors succeeded in America, and I think that it undermines much of your conclusion.
In your section titled "What explains these devastating takeovers?" you cite technology and strategic ability, but Old World diseases destroyed the communities in America before the European invaders arrived, most notably smallpox, but also measles, influenza, typhus and the bubonic plague. My reading of historians (from Charles Mann's book 1493, to Alfred W. Crosby's The Columbian Exchange and Jared Diamond's Guns Germs and Steel) leads me to conclude that the historical consensus is that the main reason for all of these takeovers was due to Old World diseases, and had relatively little to do with technology or strategy per se.
In Chapter 11 of Guns Germs and Steel, Jared Diamond analyzes the European takeovers in America you cite from the perspective of old World diseases (Here's a video from a Youtuber named CGP Grey who made a video on the same topic). The basic thesis is that Europeans had acquired immunity from these diseases, whereas people in America hadn't. From Wikipedia,
After first contacts with Europeans and Africans, some believe that the death of 90–95% of the native population of the New World was caused by Old World diseases.[43] It is suspected that smallpox was the chief culprit and responsible for killing nearly all of the native inhabitants of the Americas.
These diseases were endemic by the time that Cortes and Pizarro arrived on the continent, and therefore it seems very unlikely that their victory was achieved primarily from military and technological might. From Wikipedia again,
The Spanish Franciscan Motolinia left this description: "As the Indians did not know the remedy of the disease…they died in heaps, like bedbugs. In many places it happened that everyone in a house died and, as it was impossible to bury the great number of dead, they pulled down the houses over them so that their homes become their tombs."[46] On Cortés's return, he found the Aztec army’s chain of command in ruins. The soldiers who still lived were weak from the disease. Cortés then easily defeated the Aztecs and entered Tenochtitlán.[47] The Spaniards said that they could not walk through the streets without stepping on the bodies of smallpox victims
The effects of smallpox on Tahuantinsuyu (or the Inca empire) were even more devastating. Beginning in Colombia, smallpox spread rapidly before the Spanish invaders first arrived in the empire. The spread was probably aided by the efficient Inca road system. Within months, the disease had killed the Incan Emperor Huayna Capac, his successor, and most of the other leaders. Two of his surviving sons warred for power and, after a bloody and costly war, Atahualpa become the new emperor. As Atahualpa was returning to the capital Cuzco, Francisco Pizarro arrived and through a series of deceits captured the young leader and his best general. Within a few years smallpox claimed between 60% and 90% of the Inca population,[49] with other waves of European disease weakening them further.
The theory that disease was more important than technology is further supported empirically by the fact that Europeans were unable to conquer African tribes/civilizations until the late 19th century, long after the conquest of the New World, despite the fact that many African civilizations had similar or even lower technological capabilities compared to the Inca and Aztecs. The reason is because Africans had immunity to Old World diseases, unlike Americans. However, even in the 19th century conquests, historians often cite the development of the drug quinine, and thus immunity to disease, as one of the primary reasons why European civilizations were able to conquer African nations.
By contrast, I was only able to find one mention of smallpox in your entire post, and the place where you do mention it, you say
Smallpox sweeps through the land, killing many on all sides and causing general chaos.
If I'm reading "all sides" correctly, this is just flat-out incorrect. It killed mainly Americans.
At one point you state that during Pizarro's conquest,
The Inca empire is in the middle of a civil war and a devastating plague.
This "plague" was smallpox carried from earlier European travelers. Jared Diamond says
The reason for the civil war was that an epidemic of smallpox, spreading overland among South American Indians after its arrival with Spanish settlers in Panama and Colombia, had killed the Inca emperor Huayna Capac and most of his court around 1526, and then immediately killed his designated heir, Ninan Cuyuchi.
You may ask why there was an asymmetry: after all, didn't the New World have diseases that Europeans were not immune to? Yes, but basically only syphilis. Europeans had exposure to many infectious diseases because those diseases had been acquired from livestock, but livestock was not an important component of American civilizations in the pre-Columbian period.
One reason why disease might not be salient in descriptions of the American conquest is because until modern times, historians emphasized explanations of events in terms of human-factors, such as personalities of rulers and tendencies of groups of people. According to this source, it wasn't until the 1960s that historians started to take seriously the idea that disease was the primary culprit in the destruction of American civilizations.
There still could be an analogous situation where AI develops diseases that kills humans but not AI, but I think it's worth exploring this type of existential risk in its own category, and emphasize that this thesis does not depend on a historical precedent of conquerors having strategic or technological advantages.
This is a good critique; thank you.
I have two responses, and then a few nitpicks.
First response: Disease wasn't a part of Afonso's success. It helped the Europeans take over the Americas but did not help them take over Africa or Asia or the middle east; this suggests to me that it may have been a contributing factor but was not the primary explanation / was not strictly necessary.
Second response: Even if we decide that Cortes and Pizarro wouldn't have been able to succeed without the disease, my overall conclusion still stands. This is because disease isn't directly the cause of conquistador success, but rather indirectly, via the intermediate steps of "Chaos and disruption" and "Reduced political and economic strength." (I claim.) And the reduced strength can't have been more than, say, a 90% reduction in strength. (I claim) Suppose we think of the original conclusion as "A force with a small tech and cunning/experience advantage can take over a region 10,000 times its size." Then the modified conclusion in light of your claim about disease would be "In times of chaos and disruption, a force with a small tech and cunning/experience advantage can take over a region 1,000 times its size." This modified conclusion is, as far as I'm concerned, still almost as powerful and interesting as the original conclusion. Because "times of chaos and disruption" are pretty easy to come by. For example, it's true that the disease may have sparked the Incan civil war -- but civil wars happen pretty often anyway, historically. And when civil wars aren't happening, ordinary wars often are.
Overall, in light of your critique (and also similar things other people have said) I am going to update my original post to include a possible third factor, "Chaos & disruption / disease." I also look forward to hearing what you have to say in response to my responses.
Nitpick: The war was Cortez + allies vs. Tenochtitlan + allies. The vast majority of people on both sides were Americans. So the smallpox wreaked havoc on all sides. (Maybe I should have said "both sides" instead of "all sides")
Nitpick: If it turns out that getting sick from various diseases was what kept the Europeans out of Africa for so long, that actually supports my overall argument. (Because, imagine instead that Europeans had no problem with disease in Africa, but simply were unable to conquer much of it due to ordinary military/political reasons. Then their tech+cunning/experience advantage would have failed to be enough in that case, which makes their successes in America seem more like a fluke than a pattern explained by tech+cunning/experience. In other words, if disease wasn't a factor in Africa, that would be evidence against my claims.)
[ETA: Another way of framing my disagreement is that if you are trying to argue that small groups can take over the world, it seems almost completely irrelevant to focus on relative strategic or technological advantages in light of these historical examples. For instance, it could have theoretically been that some small technologically primitive tribe took over the world if they had some sort of immunity to disease. This would seem to imply that relative strategic advantages in Europeans vs. Americans was not that important. Instead we should focus on what ways AIs could create eg. artificial pandemics, and we could use the smallpox epidemic in America as an example of how devastating pandemics can be.]
First response: Disease wasn't a part of Afonso's success. It helped the Europeans take over the Americas but did not help them take over Africa or Asia or the middle east; this suggests to me that it may have been a contributing factor but was not the primary explanation.
That makes sense. I'm much less familiar with Afonso de Albuquerque, though my understanding is that he didn't really conquer civilizations, mostly just trading ports. I think it's safe to say that successful military campaigns are common in history, and therefore I don't find his success very unique or indicative of a future AI takeover.
Second response: Even if we decide that Cortes and Pizarro wouldn't have been able to succeed without the disease, my overall conclusion still stands.
Well, it depends. If your conclusion is that "small groups with relatively little military or strategic advantages can still take over large areas of the world" then I completely agree. If your conclusion is that, "small military or strategic advantages are by themselves often sufficient for small groups to take over large areas of the world" then I disagree. I worry your post gave the impression that the second conclusion was true.
Then the modified conclusion in light of your claim about disease would be "In times of chaos and disruption, a force with a small tech and cunning/experience advantage can take over a region 1,000 times its size." This modified conclusion is, as far as I'm concerned, still almost as powerful and interesting as the original conclusion.
A big part of my critique here is that you need to focus way more on getting the true the causal factors that lead to these historical success, because otherwise you can't use them to argue why AI is going to be anything like it.
Since disease is, in my opinion, the primary causal factor at play here, I think we should instead explore the potential for AI to engineer pandemics that kill everyone -- but that seems way different than what you were arguing.
I don't think making the thesis "in times of chaos and destruction, groups can conquer other groups" really makes the argument say much. The thing that destroyed the Incas and Aztecs was disease, not European military power, so maybe that's the lesson we should learn? Saying that merely "times of chaos" destroyed the Incas and Aztecs is tautological and not interesting.
For example, it's true that the disease may have sparked the Incan civil war -- but civil wars happen pretty often anyway, historically. And when civil wars aren't happening, ordinary wars often are.
Yes, but this Incan civil war was particularly extreme and unusual, and from the source I listed, it seems that between 60% and 90% of Incans had died. So again, determining the underlying causal factors is key to this sort of analysis.
Nitpick: The war was Cortez + allies vs. Tenochtitlan + allies. The vast majority of people on both sides were Americans. So the smallpox wreaked havoc on all sides. (Maybe I should have said "both sides" instead of "all sides")
Yeah that makes sense, but it's important to note that neither the Aztec nor the Cortez-allied Americans survived in great numbers. It was only the Spanish that were prosperous afterwards, and that's really important!
Nitpick: If it turns out that getting sick from various diseases was what kept the Europeans out of Africa for so long, that actually supports my overall argument. (Because, imagine instead that Europeans had no problem with disease in Africa, but simply were unable to conquer much of it due to ordinary military/political reasons. Then their tech+cunning/experience advantage would have failed to be enough in that case, which makes their successes in America seem more like a fluke than a pattern explained by tech+cunning/experience. In other words, if disease wasn't a factor in Africa, that would be evidence against my claims.)
I'm not sure if I understand this point well, but I think I agree. However, the quinine drug treatment for malaria was a technological advantage brought by the industrial revolution, and wasn't just some innate advantage that the Europeans eventually got.
Another way of framing my disagreement is that if you are trying to argue that small groups can take over the world, it seems almost completely irrelevant to focus on relative strategic or technological advantages in light of these historical examples. For instance, it could have theoretically been that some small technologically primitive tribe took over the world if they had some sort of immunity to disease. This would seem to imply that relative strategic advantages in Europeans vs. Americans was not that important. Instead we should focus on what ways AIs could create eg. artificial pandemics, and we could use the smallpox epidemic in America as an example of how devastating pandemics can be.
I agree that it would be good to think about how AI might create devastating pandemics. I suspect it wouldn't be that hard to do, for an AI that is generally smarter than us. However, I think my original point still stands.
I don't get why you think a small technologically primitive tribe could take over the world if they were immune to disease. Seems very implausible to me.
That makes sense. I'm much less familiar with Afonso de Albuquerque, though my understanding is that he didn't really conquer civilizations, mostly just trading ports. I think it's safe to say that successful military campaigns are common in history, and therefore I don't find his success very unique or indicative of a future AI takeover.
What difference does it make whether he conquered civilizations or ports? He did a lot of conquering despite being vastly outnumbered. This shows that "on paper" stats like army size are not super useful for determining who is likely to win a fight, at least when one side has a tech+strategic advantage. (Also, Malacca at least was a civilization in its own right; it was a city-state with a much bigger population and military than Afonso had.)
I agree that successful military campaigns are common in history. I think sometimes they can be attributed to luck, or else to genius. I chose these three case studies because they are so close to each other in time and space that they didn't seem like they could be luck or genius. I admit, however, that as lucy.ea8 said in their comment, perhaps cortes+pizarro won due to disease and then we can say Afonso was lucky or genius without stretching credibility. But I don't want to do this yet, because it seems to me that even with disease factored in, "most" of the "credit" for Cortes and Pizarro's success goes to the factors I mentioned.
After all, suppose the disease reduced the on-paper strength of the Americans by 90%. They were still several orders of magnitude stronger than Cortes and Pizarro. So it's still surprising that Cortes/Pizarro won... until we factor in the technological and strategic advantages I mentioned.
I don't think making the thesis "in times of chaos and destruction, groups can conquer other groups" really makes the argument say much. The thing that destroyed the Incas and Aztecs was disease, not European military power, so maybe that's the lesson we should learn? Saying that merely "times of chaos" destroyed the Incas and Aztecs is tautological and not interesting.
But the civilizations wouldn't have been destroyed without the Spaniards. (I might be wrong about this, but... hadn't the disease mostly swept through Inca territory by the time Pizarro arrived? So clearly their civilization had survived.)
I think I am somewhat close to being convinced by your criticism, at least when phrased in the way you just did: "your thesis is trivial!" But I'm not yet convinced, because of my argument about the 90% reduction. (I keep making the same argument basically in response to all your points; it is the crux for me I think.)
Yeah that makes sense, but it's important to note that neither the Aztec nor the Cortez-allied Americans survived in great numbers. It was only the Spanish that were prosperous afterwards, and that's really important!
Even after the disease took its toll, the Spaniards were vastly outnumbered by the Americans. Analogy: Suppose Coronavirus wipes out 90% of the world's population of asthmatic smoker 90+ yr old men. And suppose that also, the mathematics department at MIT produces more published theorems in 2021 than that entire demographic. The first fact is not the primary explanation for the second fact. Even if Coronavirus didn't happen and that demographic was not reduced by 90%, the MIT mathematics department still would have produced more published theorems. (And even if this last thing isn't true -- even if by some mathematical coincidence that 90% factor would make the difference -- it still wouldn't be fair to say that disease is the primary factor, it is clearly much less important than e.g. age, specialization, health, etc.)
I agree that it would be good to think about how AI might create devastating pandemics. I suspect it wouldn't be that hard to do, for an AI that is generally smarter than us. However, I think my original point still stands.
It's worth clarifying exactly what "original point" stands because I'm currently unsure.
I don't get why you think a small technologically primitive tribe could take over the world if they were immune to disease. Seems very implausible to me.
Sorry, I meant to say, "Were immune to diseases that were currently killing everyone else." If everyone is dying around you, then your level of technology doesn't really matter that much. You just wait for your enemy to die and then settle the land after they are gone. This is arguably what Europeans did in America. My point is that by focusing on technology, you are missing the main reason for the successful conquest.
But I don't want to do this yet, because it seems to me that even with disease factored in, "most" of the "credit" for Cortes and Pizarro's success goes to the factors I mentioned.
After all, suppose the disease reduced the on-paper strength of the Americans by 90%. They were still several orders of magnitude stronger than Cortes and Pizarro. So it's still surprising that Cortes/Pizarro won... until we factor in the technological and strategic advantages I mentioned.
I feel like you don't actually have a civilization if 90% of your people died. I think it's more fair to say that when 90% of your people die, your civilization basically stops existing rather than just being weakened. For example, I can totally imagine an Incan voyage to Spain conquering Madrid if 90% of the Spanish died. Their chain of command would be in complete shambles. It wouldn't just be like some clean 90% reduction in GDP with everything else held constant.
But the civilizations wouldn't have been destroyed without the Spaniards. (I might be wrong about this, but... hadn't the disease mostly swept through Inca territory by the time Pizarro arrived? So clearly their civilization had survived.)
I think I am somewhat close to being convinced by your criticism, at least when phrased in the way you just did: "your thesis is trivial!" But I'm not yet convinced, because of my argument about the 90% reduction. (I keep making the same argument basically in response to all your points; it is the crux for me I think.)
Look, if 90% of a country dies of a disease, and then the surviving 10% become engulfed in a civil war, and then some military group who is immune to the disease comes in and takes the capital city during this all, don't you think it's very misleading to conclude "A small group of people with a slight military advantage can take over a large civilization" without heavily emphasizing the whole 90% of people dying of a disease part? This is the heart of my critique.
My original point was that sometimes, a small group can reliably take over a large region despite being vastly outnumbered and outgunned, having only slightly better tech and cunning, knowing very little about the region to be conquered, and being disunited. This is in the context of arguments about how much of a lead in AI tech one needs to have to take over the world, and how big of an entity one needs to be to do it (e.g. can a rogue AI do it? What about a corporation? A nation-state?) Even with your point about disease, it still seems I'm right about this, for reasons I've mentioned (the 90% argument)
I really don't think the disease thing is important enough to undermine my conclusion. For the two reasons I gave: One, Afonso didn't benefit from disease, and two, the 90% argument: Suppose there was no disease but instead the Aztecs and Incas were 90% smaller in population and also in the middle of civil war. Same result would have happened, and it still would have proved my point.
I don't think a group of Incans in Spain could have taken it over if 90% of the Spaniards were dying of disease. I think they wouldn't have had the technology or experience necessary to succeed.
Is this a fair description of your disagreement re the 90% argument?
Daniel thinks that a 90% reduction in the population of a civilization corresponds to a ~90% reduction in their power/influentialness. Because the Americans so greatly outnumbered the Spanish, this ten-fold reduction in power/influentialness doesn’t much alter the conclusion.
Matthew thinks that a 90% reduction in the population of a civilization means that “you don’t really have a civilization”, which I interpret to mean something like a ~99.9%+ reduction in the power/influentialness of a civilization, which occurs mainly through a reduction in their ability to coordinate (e.g. “chain of command in ruins”). This is significant enough to undermine the main conclusion.
If this is accurate, would a historical survey of the power/influentialness of civilisations after they lose 90% of the population (inasmuch as these cases exist) resolve the disagreement?
For my part, I think you summarized my position fairly well. However, after thinking about this argument for another few days, I have more points to add.
- Disease seems especially likely to cause coordination failures since it's an internal threat rather than an external threat (which unlike internal threats, tend to unite empires). We can compare the effects of the smallpox epidemic in the Aztec and Inca empires alongside other historical diseases during wartime, such as the Plauge of Athens which arguably is what caused Athens to lose the Peloponnesian War.
- Along these same lines, the Aztec/Inca didn't have any germ theory of disease, and therefore didn't understand what was going on. They may have thought that the gods were punishing them for some reason, and therefore they probably spent a lot of time blaming random groups for the catastrophe. We can contrast these circumstances to eg. the Paraguayan War which killed up to 90% of the male population, but people probably had a much better idea what was going on and who was to blame, so I expect that the surviving population had an easier time coordinating.
- A large chunk of the remaining population likely had some sort of disability. Think of what would happen if you got measles and smallpox in the same two year window: even if you survived it probably wouldn't look good. This means that the pure death rate is an underestimate of the impact of a disease. The Aztecs, for whom "only" 40 percent died of disease, were still greatly affected
It killed many of its victims outright, particularly infants and young children. Many other adults were incapacitated by the disease – because they were either sick themselves, caring for sick relatives and neighbors, or simply lost the will to resist the Spaniards as they saw disease ravage those around them. Finally, people could no longer tend to their crops, leading to widespread famine, further weakening the immune systems of survivors of the epidemic. [...] a third of those afflicted with the disease typically develop blindness.
I accept that these points are evidence in your favor. Here are some more of my own:
--Smallpox didn't hit the Aztecs until Cortes had already killed the Emperor and allied with the Tlaxcalans, if I'm reading these summaries correctly. (I really should go read the actual books...) So it seems that Cortes did get really far on the path towards victory without the help of disease. More importantly, there doesn't seem to be any important difference in how people treated Cortes before or after the disease. They took him very seriously, underestimated him, put too much trust in him, allied with him, etc. before the disease was a factor.
--When Pizarro arrived in Inca lands, the disease had already swept through, if I'm reading these stories right. So the period of most chaos and uncertainty was over; people were rebuilding and re-organizing.
--Also, it wasn't actually a 90% reduction in population. It was more like a 50% reduction at the time, if I am remembering right. (Later epidemics would cause further damage, so collectively they were worse than any other plague in history.) This is comparable to e.g. the Black Death in Europe, no? But the Black Death didn't result in the collapse of most civilizations who went through it, nor did it result in random small groups of adventurers taking over governments, I predict. (I haven't actually read up on the history of it)
Thanks Alexis, this seems like an accurate description to me. Strong-upvoted, partly because I want to reward people for doing these sorts of summary-and-distillation stuff.
As for your question, hmm, I'm not sure. I tentatively say yes, but my hesitations are (1) cases where 90% of the population dies are probably very rare, and (2) how would we measure power anyway? Presumably most civilizations that lose 90% of their population do end up conquered by someone else pretty quickly, since most civilizations aren't 10x more powerful than all their neighbors.
I think the crux is this business about the chain of command. Cortez and Pizarro succeeded by getting Americans to ally with them and/or obey them. The crux is, would they have been able to do this as well or mostly as well without the disease? I think that reading a bunch of books on what happened might more or less answer this question.
For example, maybe the books will say that the general disarray caused by the disease created a sense of desperation and confusion in the people which led them to be open to the conquistador's proposals when otherwise they would have dismissed them. In which case, I concede defeat in this disagreement. Or maybe the books will say that if only the conquistadors had been outnumbered even more, they would have lost.
But what I predict is that the books will say that, for the most part, the reasons why people allied with Cortes and Pizarro had more to do with non-disease considerations: "Here is this obviously powerful representative of an obviously powerful faraway empire, wielding intriguing technology that we could benefit from. There is our hated enemy, Tenochtitlan, who has been oppressing us for decades. Now is our chance to turn the tables on our oppressors!" Similarly, I predict that the reason why the emperors allowed the conquistadors to get close enough to ambush them have little to do with disease and more to do with, well, just not predicting that the conquistadors would have the will or capability to do that. Moreover I predict that adding even more native warriors (due to the disease not happening) wouldn't have caused the conquistadors to lose. After all, roughly half of those warriors would be added to the conquistador's side...
So I highly recommend that someone who doesn't have a dog in this fight go read some books on Cortes and Pizarro and then report back!
Update: I do think it would be good to look at the Black Death in Europe and see whether there were similar political "upsets" where a small group of outsiders took over a large region in the turmoil. I predict that there mostly weren't; if it turns out this did happen a fair amount, then I agree that is good evidence that disease was really important.
I really don't think the disease thing is important enough to undermine my conclusion. For the two reasons I gave: One, Afonso didn't benefit from disease
This makes sense, but I think the case of Afonso is sufficiently different from the others that it's a bit of a stretch to use it to imply much about AI takeovers. I think if you want to make a more general point about how AI can be militarily successful, then a better point of evidence is a broad survey of historical military campaigns. Of course, it's still a historically interesting case to consider!
two, the 90% argument: Suppose there was no disease but instead the Aztecs and Incas were 90% smaller in population and also in the middle of civil war. Same result would have happened, and it still would have proved my point.
Yeah but why are we assuming that they are still in the civil war? Call me out if I'm wrong here, but your thesis now seems to be: if some civilization is in complete disarray, then a well coordinated group of slightly more advanced people/AI can take control of the civilization.
This would be a reasonable thesis, but it doesn't shed too much light on AI takeovers. The important part lies in the "if some civilization is in complete disarray" conditional, and I think it's far from obvious that AI will emerge in such a world, unless some other more important causal factor already occurred that gave rise to the massive disarray in the first place. But even in that case, don't you think we should focus on that thing that caused the disarray instead?
Again, I certainly agree that it would be good to think about things that could cause disarray as well. Like you said, maybe an AI could easily arrange for there to be a convenient pandemic at about the time it makes its move...
And yeah, in light of your pushback I'm thinking of moderating my thesis to add the "disarray background condition" caveat. (I already edited the OP)This does weaken the claim, but not much, I think, because the sort of disarray needed is relatively common, I think. For purposes of Cortes and Pizarro takeover, what mattered was that they were able to find local factions willing to ally with them to overthrow the main power structures. The population count wasn't super relevant because, disease or no, it was several orders of magnitude more than Cortez & Pizarro had. And while it's true that without the disease they may have had a harder time finding local factions willing to ally with them, it's not obviously true, and moreover there are plenty of ordinary circumstances (ordinary civil wars, ordinary periods of unrest and rebellion, ordinary wars between great powers) that lead to the same result: Local factions being willing to ally with an outsider to overthrow the main power structure.
This conversation has definitely made me less confident in my conclusion. I now think it would be worth it for me (or someone) to go do a bunch of history reading, to evaluate these debates with more information.
Some more thoughts that came to me after I posted this:
1. I should have mentioned who I take myself to be responding to. I'm responding to claims I've heard made by various people (Katja Grace and Paul Christiano, to name two very smart examples) that I summarize as follows: "For one entity to take over the world, it needs more power than the rest of the world combined. That means it must either start out with a lot of power -- e.g. a major nation-state like China or USA -- or grow in power much faster than the rest of the world for a while. The latter possibility could happen if an intelligence explosion occurred in one local area, but this is unlikely; the intelligence explosion is more likely to be distributed fairly broadly."
2. I think these case studies are also historical precedents for "Treacherous Turns." Cortes and Pizarro were certainly very treacherous. (I don't think Afonso was, but I'd have to reread the stories.)
3. I have vague, conspiracy-theory-esque worries that actually the conquistadors really did just all get lucky, and that the explanation for this is some sort of anthropic selection effect. I don't think this is plausible though. But boy do they seem lucky when you read the stories.
4. Cortes, Pizarro, and Afonso all benefited from claiming to represent a powerful European state that would be sending more reinforcements soon. (In several cases this was blatant lying, e.g. Cortes was insubordinate and the reinforcements were out to arrest him!) Does this have an analogy in the AI case? I think it does, actually. An AI could claim that its recommendations are mutually beneficial and will seem obvious in retrospect and that future AIs we build will all say the same thing, so we either get along with this first AI or we resign ourselves to not making use of any AI in the future. (And it may actually be true that other AIs we build will say the same thing as the first AI, though not necessarily because its recommendations are mutually beneficial and obvious in retrospect!)
5. I overall am now fairly convinced that an AI which is generally more intelligent than us could take over the world, even if it isn't superintelligent. Even if it is to our best and brightest what our best and brightest are to our average folks -- which isn't that big of a difference -- I think it could take over the world. Or maybe a small group of such agents could do so. After all, I say: The difference between the conquistadors and their conquered victims is surely smaller than that.
(I am the author)
I still like & endorse this post. When I wrote it, I hadn't read more than the wiki articles on the subject. But then afterwards I went and read 3 books (written by historians) about it, and I think the original post held up very well to all this new info. In particular, the main critique the post got -- that disease was more important than I made it sound, in a way that undermined my conclusion -- seems to have been pretty wrong. (See e.g. this comment thread, these follow up posts)
So, why does it matter? What contribution did this post make? Well, at the time -- and still now, though I think I've made a dent in the discourse -- quite a lot of people I respect (such as people at OpenPhil) seemed to think unaligned AGI would need god-like powers to be able to take over the world -- it would need to be stronger than the rest of the world combined! I think this is based on a flawed model of how takeover/conquest works, and history contains plenty of counterexamples to the model. The conquistadors are my favorite counterexample from my limited knowledge of history. (The flawed model goes by the name of "The China Argument," at least in my mind. You may have heard the argument before -- China is way more capable than the most capable human, yet it can't take over the world; therefore AGI will need to be way way more capable than the most powerful human to take over the world.)
Needless to say, this is a somewhat important crux, as illustrated by e.g. Joe Carlsmith's report, which assigns a mere 40% credence to unaligned APS-AI taking over the world even conditional on it escaping and seeking power and managing to cause at least a trillion dollars worth of damage.
(I've also gotten feedback from various people at OpenPhil saying that this post was helpful to them, so yay!)
I've since written a sequence of posts elaborating on this idea: Takeoff and Takeover in the Past and Future. Alas, I still haven't written the capstone posts in the sequence, the posts that'll tie it all together. I've got some new arguments and models to add too!
I think that if this post makes the review (and if it's allowed) I'll want to revise it as heavily as the rules allow, to include more analysis of the sort I've been doing in the sequence and plan to do in the future capstone post(s).
However, I don't think this is the whole explanation. The technological advantage of the conquistadors was not overwhelming.
With regard to the Americas at least, I just happened to read this article by a professional military historian, who characterizes the Native American military technology as being "thousands of years behind their Old World agrarian counterparts", which sounds like the advantage was actually rather overwhelming.
There is a massive amount of literature to explain what is sometimes called ‘the Great Divergence‘ (a term I am going to use here as valuable shorthand) between Europe and the rest of the world between 1500 and 1800. Of all of this, most readers are likely only to be familiar with one work, J. Diamond’s Guns, Germs and Steel (1997), which is unfortunate because Diamond’s model of geographic determinism is actually not terribly well regarded in the debate (although, to be fair, it is still better than some of the truly trash nationalistic nonsense that gets produced on this topic). Diamond asks the Great Divergence question with perhaps the least interesting framing: “Why Europe and not the New World?” and so we might as well get that question out of the way first.
I am well aware that when EU4 was released, this particular question – and generally the relative power of New World societies as compared to Old World societies – was a point of ferocious debate among fans (particularly on Paradox’s own forums). What makes this actually a less central question (though still an important one) is that the answer is wildly overdetermined. That is to say, any of these causes – the germs, the steel (through less the guns; Diamond’s attention is on the wrong developments there), but also horses, ocean-going ships, and dense, cohesive, disciplined military formations would have been enough in isolation to give almost any complex agrarian Old-World society military advantages which were likely to prove overwhelming in the event. The ‘killer technologies’ that made the conquest of the New World possible were (apart from the ships) old technologies in much of Afroeurasia; a Roman legion or a Han Chinese army of some fifteen centuries earlier would have had many of the same advantages had they been able to surmount the logistical problem of actually getting there. In the face of the vast shear in military technology (though often not in other technologies) which put Native American armies thousands of years behind their Old World agrarian counterparts, it is hard not to conclude that whatever Afroeurasian society was the first to resolve the logistical barriers to putting an army in the New World was also very likely to conquer it.
(On these points, see J.F. Guilmartin, “The Cutting Edge: An Analysis of the Spanish Invasion and Overthrow of the Inca Empire, 1532-1539,” in Transatlantic Encounters: European and Andeans in the Sixteenth Century, eds. K. J. Andrien and R. Adorno (1991) and W.E. Lee, “The Military Revolution of Native North America: Firearms, Forts and Politics” in Empires and Indigenes: Intercultural Alliance, Imperial Expansion and Warfare in the Early Modern World, eds. W.E. Lee (2011). Both provide a good sense of the scale of the ‘technological shear’ between old world and new world armies and in particular that the technologies which were transformative were often not new things like guns, but very old things, like pikes, horses and metal axes.)
With regard to the Indian Ocean, he writes:
the Portuguese cartaz-system (c. 1500-c. 1700) [was] the main way that the Portuguese and later European powers wrested control over trade in the Indian Ocean; it only worked because Portuguese warships were functionally unbeatable by anything else afloat in the region due to differences in local styles of shipbuilding).
I don't really see this as in strong conflict with what I said. I agree that technology is the main factor; I said it was also "strategic and diplomatic cunning;" are you suggesting that it wasn't really that at all and that if Cortez had gifted his equipment to 500 locals they would have been just as successful at taking over as he was? I could be convinced of this I suppose.
I think technological advantage - specifically sailing technology - probably played a much larger role in Afonso's takeover than it would seem from a quick read. Key pieces:
- Monsoons
- Lateen sail
Monsoons: wind around India blows consistently Southwest for half the year, and Northeast for the other half. IIRC from Braudel, this made trade in the Indian ocean highly predictable: everyone sailed with the wind at their back and ran consistent one-year circuits. As you mention:
The Indian Ocean contained most of the world's trade at the time, since it linked up the world's biggest and wealthiest regions.
I'd guess that the monsoons were probably a bigger factor here than vicinity to wealthy regions. In particular:
Europe is just coming out of the Middle Ages and does not have an obvious technological advantage over India or China or the Middle East, and has an obvious economic disadvantage.
Europe had less total wealth (because it had a smaller population) and was behind technologically in some ways (e.g. metallurgy), but even in the 15th-16th century Europe was considered "wealthy" on a per-capita basis. In particular, Europe had much more per-capita capital goods, even before the industrial revolution - especially mills and machinery. Braudel covers a lot of this.
Anyway, monsoons. Consistent wind direction, with an annual cycle. That makes the lateen sail a major strategic advantage: Portugese ships would have been able to tack upwind, a technique which was basically unheard of in the Indian ocean at the time. (On top of that, the Portugese were happy to sail in open ocean at that time and were accustomed to navigating away from land - unlike the Indian ocean locals. Again, Braudel talks about this a fair bit.) So the local navies were presumably stuck at one end of the ocean for six months, while the Portugese had free reign to sail around wherever they wanted. And to top it all off, even if the local navies did manage to catch them, the Portugese could just sail out to open ocean, and the locals wouldn't want to follow.
Now combine that with supply: throughout most of history, a single ship could carry as much supplies as about 4000 horses (source: Logistics of the Macedonian Army). For any island garrisons, or for garrisons surrounded by desert, horses wouldn't even be an option. Thus the importance of naval dominance even for land wars in premodern times: an overland supply train was extremely expensive at best, and often entirely infeasible. Control the water, and the enemy starved.
Put all that together, and Afonso's plan looks less ridiculously ambitious. They had a technological advantage which was perfectly suited to the problem.
Update: According to the wiki article on lateen sails, they existed for several hundred years in the Mediterranean before spreading to the Atlantic, and the Nile, and then finally they arrived in the Indian ocean with the Portuguese, at which point the locals quickly adopted it on their vessels also. (Within 20 years!)
What the hell? Why did it take so long? If it was so good that it gave a huge advantage, such that everyone copied the design within two decades of the Portuguese arrival... why did no one notice this for almost a thousand years? Surely there were travelers who sailed on both the Med. and Red seas, for example. Surely the Ottomans and Mamelukes, who maintained fleets in both the Med. and the Indian Ocean, should have been able to realize that the lateen sail was a thing and would be useful? (Especially since being able to sail against the wind seems super useful precisely when the wind doesn't change direction very much, e.g. in monsoon-regions like the Indian Ocean) Also apparently the pacific islanders independently invented the lateen sail, yet it didn't spread from there to the Indian Ocean either. I am very confused.
Thanks for this, I didn't know about that stuff. Perhaps I should read Braudel. This makes me more confident that technology was a necessary factor. If I were motivated to do so, I'd read through the stories of Afonso's success in more detail and try to see whether open-sea sailing and/or sailing against the wind played a factor.
Braudel is both long and dense, and I wouldn't recommend the second two volumes at all, but the first volume is probably the single best history book I've read. Beware that his understanding of economics is pretty poor - trust his facts, but be wary of his interpretations.
Nice post. Were there any sources besides Wikipedia that you found especially helpful when researching this post?
This post was based on very little research; all I did was read the wiki pages. So it's possible that a real historian (or a real history book) would yield different conclusions. However, I am fairly confident this won't happen for the main conclusion of my post: that you don't need a god-like technological advantage for a tiny group to have a good shot at quickly taking over a large region.
I forgot to give an update: Now I have read a handful of real history books on the subject, and I think the original post still stands strong.
Based on what I recall reading about Pizzaro's conquest, I feel you might be underestimating the importance of horses. It took centuries for European powers to figure out how to break a heavy cavalry charge with infantry; the amerindians didn't have the time to figure it out (see various battles where small cavalry forces routed thousands of troops). Once they had got more used to horses, later Inca forces (though much diminished) were more able to win open battles against the Spanish.
Maybe this was the problem for these empires: they were used to winning open battles, but were presented with a situation where only irregular warfare or siege defences could win. They reacted as an empire, when they should have been reacting as a recalcitrant province.
Mmm, interesting. I'm now reading a 1400-page history book on the subject (after all the attention my post got, I figured I should read more than just a bunch of wiki pages!) so we'll know one way or another soon enough. Thanks for the tip.
Thanks for your research, especially the Afonso stuff. One question for that: were these empires used to gaining/losing small pieces of territory? ie did they really dedicate all their might to getting these ports back, or did they eventually write them off as minor losses not worth the cost of fighting (given Portuguese naval advantages)?
Good question; I'll find out. Malacca at least was a city-state, so the Portuguese attack was an existential threat.
Here's what I'll be putting in the Alignment Newsletter about this piece. Let me know if you spot inaccuracies or lingering disagreement regarding the opinion section.
Summary:
This post lists three historical examples of how small human groups conquered large parts of the world, and shows how they are arguably precedents for AI takeover scenarios. The first two historical examples are the conquests of American civilizations by Hernán Cortés and Francisco Pizarro in the early 16th century. The third example is the Portugese capture of key Indian Ocean trading ports, which happened at roughly the same time as the other conquests. Daniel argues that technological and strategic advantages were the likely causes of these European victories. However, since a European technological advantage was small in this period, we might expect that an AI coalition could similarly take over a large portion of the world, even without a large technological advantage.
Opinion:
In a comment, I dispute the claimed reasons for why Europeans conquered American civilizations. I think that a large body of historical literature supports the conclusion that American civilizations fell primarily because of their exposure to diseases which they lacked immunity to, rather than because of European military power. I also think that this helps explain why Portugal was "only" able to capture Indian Ocean trading ports during this time period, rather than whole civilizations. I think the primary insight here should instead be that pandemics can kill large groups of humans, and therefore it would be worth exploring the possibility that AI systems use pandemics as a mechanism to kill large numbers of biological humans.
Thanks! Well, I still disagree with your opinion on it, for reasons mentioned above. To the point about "only" conquering ports, well, I think my explanations fit fine with that too -- the technological and experience advantages that (I claim) enabled Afonso to win were primarily naval in nature. Later, other Europeans would come along with other advantages, and they would conquer India, Persia, Vietnam, etc., evidence that while disease was a contributing factor (I certainly am not denying it helped!) it wasn't so important a factor as to render my conclusion invalid (my conclusion, again, is that a moderate technological and strategic advantage can enable a small group to take over a large region.)
Later, other Europeans would come along with other advantages, and they would conquer India, Persia, Vietnam, etc., evidence that while disease was a contributing factor (I certainly am not denying it helped!) it wasn't so important a factor as to render my conclusion invalid (my conclusion, again, is that a moderate technological and strategic advantage can enable a small group to take over a large region.)
Europeans conquered places such as India, but that was centuries later, after they had a large technological advantage, and they also didn't come with just a few warships either: they came with vast armadas. I don't see why that supports the point that a small group can take over a large region?
The vast armadas were the result of successful colonization, not the cause of it. For example, a key battle that the British EIC won (enabling them to take over their first major territory) was the battle of Plassey, and they were significantly outnumbered during it.
Fair point about the large technological advantage, but... actually it still wasn't that large? I don't know, I'd have to look into it more, but my guess is that the tech advantage of the EIC over the Nawab at Plassey, to use the same example, was smaller than the tech advantage of Cortes and Pizarro over the Americans.
I should go find out how many men the EIC had when it conquered India. I'm betting that the answer is "Far fewer than India had." And also, yeah, didn't the British steal rocket technology from India? (Mysore, I think?) That's one military important technology that they were actually behind in.
Promoted to curated: I really like this post. I already linked to it two times, and it clearly grounds some examples that I've seen people use informally in AI Alignment discussions in a way that will hopefully create a common reference, and allow us to analyze this kind of argument in much more detail.
NEW EDIT: After reading three giant history books on the subject, I take back my previous edit. My original claims were correct.
Could you edit this comment to add which three books you're referring to?
"The Conquest of Mexico and the Conquest of Peru"
"1493"
and... the one about Afonso I forget the title... it might have been this History of Portugal : Marques, Antonio Henrique R. de Oliveira : Free Download, Borrow, and Streaming : Internet Archive or this https://www.amazon.com/gp/search?index=books&tag=NYTBSREV-20&field-keywords=Conquerors%3A+How+Portugal+Forged+the+First+Global+Empire+Roger+Crowley
I also ended up reading "1491" and "1492."
Another thought:
Japan was in a similar predicament in the mid-1850s, but successfully managed not only to survive but to thrive, modernizing their economy and military to Western levels without suffering regime change in the process.
What explains their success? It's not that the technological gap between them and Western powers was smaller than the technological gap between the conquistadors and their victims. (I think.) Rather, it's that there was no "diplomatic and strategic cunning" gap, and that was because there was no experience gap. Japan had been economically and militarily isolated from the rest of the world, but it had been watching developments around the world with keen interest via reports from the few Dutch and Chinese visitors it allowed into the Nagasaki port. For example, the Japanese government knew about Perry's expedition before it arrived, thanks to a tip from the Dutch. So when Perry arrived to force Japan open, the Japanese government was as well-informed as he was.
Fun fact: I predicted, after writing the above paragraph, that the Japanese emperor would not have gone anywhere near Perry for the negotiation. (Remember Cortes and Pizarro?) Sure enough, this was correct -- well actually the emperor was a figurehead at the time, but the man in charge stayed well away from the American visitors.
To be clear, I don't want to lean too heavily on this explanation. I think it's probably not even the main explanation; I wouldn't be surprised if more unity within Japan, or less rapaciousness within America at the time (or at least in Perry?), were bigger factors. I haven't done much reading on the topic. But I would bet that the lack of an experience gap played some role at least.
Crossposted from AI Impacts.
Epistemic status: I am not a historian, nor have I investigated these case studies in detail. I admit I am still uncertain about how the conquistadors were able to colonize so much of the world so quickly. I think my ignorance is excusable because this is just a blog post; I welcome corrections from people who know more. If it generates sufficient interest I might do a deeper investigation. Even if I’m right, this is just one set of historical case-studies; it doesn’t prove anything about AI, even if it is suggestive. Finally, in describing these conquistadors as “successful,” I simply mean that they achieved their goals, not that what they achieved was good.
Summary
In the span of a few years, some minor European explorers (later known as the conquistadors) encountered, conquered, and enslaved several huge regions of the world. That they were able to do this is surprising; their technological advantage was not huge. (This was before the scientific and industrial revolutions.) From these cases, I think we learn that it is occasionally possible for a small force to quickly conquer large parts of the world, despite:
Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.
EDIT: In light of good pushback from people (e.g. Lucy.ea8 and e.g. Matthew Barnett) about the importance of disease, I think one should probably add a caveat to the above: "In times of chaos & disruption, at least."
NEW EDIT: After reading three giant history books on the subject, I take back my previous edit. My original claims were correct.
Three shocking true stories
I highly recommend you read the wiki pages yourself; otherwise, here are my summaries:
Cortés: [wiki] [wiki]
Pizarro [wiki] [wiki]
Afonso [wiki] [wiki] [wiki]
Some comparisons and contrasts:
What explains these devastating conquests?
Wrong answer: I cherry-picked my case studies.
History is full of incredibly successful conquerors: Alexander the Great, Ghenghis Khan, etc. Perhaps some people are just really good at it, or really lucky, or both.
However: Three incredibly successful conquerors from the same tiny region and time period, conquering three separate empires? Followed up by dozens of less successful but still very successful conquerors from the same region and time period? Surely this is not a coincidence. Moreover, it’s not like the conquistadors had many failed attempts and a few successes. The Aztec and Inca empires were the two biggest empires in the Americas, and there weren’t any other Indian Oceans for the Portuguese to fail at conquering.
Fun fact: I had not heard of Afonso before I started writing this post this morning. Following the Rule of Three, I needed a third example and I predicted on the basis of Cortes and Pizarro that there would be other, similar stories happening in the world at around that time. That’s how I found Afonso.
Right answer: Technology
However, I don't think this is the whole explanation. The technological advantage of the conquistadors was not overwhelming.
Whatever technological advantage the conquistadors had over the existing empires, it was the sort of technological advantage that one could acquire before the Scientific and Industrial revolutions. Technology didn't change very fast back then, yet Portugal managed to get a lead over the Ottomans, Egyptians, Mughals, etc. that was sufficient to bring them victory. On paper, the Aztecs and Spanish were pretty similar: Both were medieval, feudal civilizations. I don't know for sure, but I'd bet there were at least a few techniques and technologies the Aztecs had that the Spanish didn't. And of course the technological similarities between the Portuguese and their enemies were much stronger; the Ottomans even had access to European mercenaries! Even in cases in which the conquistadors had technology that was completely novel -- like steel armor, horses, and gunpowder were to the Aztecs and Incas -- it wasn't god-like. The armored soldiers were still killable; the gunpowder was more effective than arrows but limited in supply, etc.
(Contrary to popular legend, neither Cortés nor Pizarro were regarded as gods by the people they conquered. The Incas concluded pretty early on that the Spanish were mere men, and while the idea did float around the Aztecs for a bit the modern historical consensus is that most of them didn't take it seriously.)
Ask yourself: Suppose Cortés had found 500 local warriors, gave them all his equipment, trained them to use it expertly, and left. Would those local men have taken over all of Mexico? I doubt it. And this is despite the fact that they would have had much better local knowledge than Cortés did! Same goes for Pizarro and Afonso. Perhaps if he had found 500 local warriors led by an exceptional commander it would work. But the explanation for the conquistador’s success can’t just be that they were all exceptional commanders; that would be positing too much innate talent to occur in one small region of the globe at one time.
Right answer: Strategic and diplomatic cunning
This is my non-expert guess about the missing factor that joins with technology to explain this pattern of conquistador success.
They didn't just have technology; they had effective strategy and they had effective diplomacy. They made long-term plans that worked despite being breathtakingly ambitious. (And their short-term plans were usually pretty effective too, read the stories in detail to see this.) Despite not knowing the local culture or history, these conquistadors made surprisingly savvy diplomatic decisions. They knew when they could get away with breaking their word and when they couldn't; they knew which outrages the locals would tolerate and which they wouldn’t; they knew how to convince locals to ally with them; they knew how to use words to escape militarily impossible situations… The locals, by contrast, often badly misjudged the conquistadors, e.g. not thinking Pizarro had the will (or the ability?) to kidnap the emperor, and thinking the emperor would be safe as long as they played along.
This raises the question, how did they get that advantage? My answer: they had experience with this sort of thing, whereas locals didn't. Presumably Pizarro learned from Cortés' experience; his strategy was pretty similar. (See also: the prior conquest of the Canary Islands by the Spanish). In Afonso's case, well, the Portuguese had been sailing around Africa, conquering ports and building forts for more than a hundred years.
Lessons I think we learn
I think we learn that:
It is occasionally possible for a small force to quickly conquer large parts of the world, despite:
Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.
EDIT: In light of good pushback from people (e.g. Lucy.ea8 and e.g. Matthew Barnett) about the importance of disease, I think one should probably add a caveat to the above: "In times of chaos & disruption, at least."
Having only a minuscule fraction of the world's resources and power
In all three examples, the conquest was more or less completed without support from home; while Spain/Portugal did send reinforcements, it wasn't even close to the entire nation of Spain/Portugal fighting the war. So these conquests are examples of non-state entities conquering states, so to speak. (That said, their claim to represent a large state may have been crucial for Cortes and Pizarro getting audiences and respect initially.) Cortés landed with about a thousandth the troops of Tenochtitlan, which controlled a still larger empire of vassal states. Of course, his troops were better equipped, but on the other hand they were also cut off from resupply, whereas the Aztecs were in their home territory, able to draw on a large civilian population for new recruits and resupply.
The conquests succeeded in large part due to diplomacy. This has implications for AI takeover scenarios; rather than imagining a conflict of humans vs. robots, we could imagine humans vs. humans-with-AI-advisers, with the latter faction winning and somehow by the end of the conflict the AI advisers have managed to become de facto rulers, using the humans who obey them to put down rebellions by the humans who don't.
Having technology + diplomatic and strategic skill that is better but not that much better
As previously mentioned, the conquistadors didn’t enjoy god-like technological superiority. In the case of Afonso the technology was pretty similar. Technology played an important role in their success, but it wasn’t enough on its own. Meanwhile, the conquistadors may have had more diplomatic and strategic cunning (or experience) than the enemies they conquered. But not that much more--they are only human, after all. And their enemies were pretty smart.
In the AI context, we don't need to imagine god-like technology (e.g. swarms of self-replicating nanobots) to get an AI takeover. It might even be possible without any new physical technologies at all! Just superior software, e.g. piloting software for military drones, targeting software for anti-missile defenses, cyberwarfare capabilities, data analysis for military intelligence, and of course excellent propaganda and persuasion.
Nor do we need to imagine an AI so savvy and persuasive that it can persuade anyone of anything. We just need to imagine it about as cunning and experienced relative to its enemies as Cortés, Pizarro, and Afonso were relative to theirs. (Presumably no AI would be experienced with world takeover, but perhaps an intelligence advantage would give it the same benefits as an experience advantage.) And if I’m wrong about this explanation for the conquistador’s success--if they had no such advantage in cunning/experience--then the conclusion is even stronger.
Additionally, in a rapidly-changing world that is undergoing slow takeoff, where there are lesser AIs and AI-created technologies all over the place, most of which are successfully controlled by humans, AI takeover might still happen if one AI is better, but not that much better, than the others.
Having very little data about the world when the conquest begins
Cortés invaded Mexico knowing very little about it. After all, the Spanish had only realized the Americas existed two decades prior. He heard rumors of a big wealthy empire and he set out to conquer it, knowing little of the technology and tactics he would face. Two years later, he ruled the place.
Pizarro and Afonzo were in better epistemic positions, but still, they had to learn a lot of important details (like what the local power centers, norms, and conflicts were, and exactly what technology the locals had) on the fly. But they were good at learning these things and making it up as they went along, apparently.
We can expect superhuman AI to be good at learning. Even if it starts off knowing very little about the world -- say, it figured out it was in a training environment and hacked its way out, having inferred a few general facts about its creators but not much else -- if it is good at learning and reasoning, it might still be pretty dangerous.
Being disunited
Cortés invaded Mexico in defiance of his superiors and had to defeat the army they sent to arrest him. Pizarro ended up fighting a civil war against his fellow conquistadors in the middle of his conquest of Peru. Afonzo fought Greek mercenaries and some traitor Portuguese, conquered Malacca against the orders of a rival conquistador in the area, and was ultimately demoted due to political maneuvers by rivals back home.
This astonishes me. Somehow these conquests were completed by people who were at the same time busy infighting and backstabbing each other!
Why was it that the conquistadors were able to split the locals into factions, ally with some to defeat the others, and end up on top? Why didn't it happen the other way around: some ambitious local ruler talks to the conquistadors, exploits their internal divisions, allies with some to defeat the others, and ends up on top?
I think the answer is partly the "diplomatic and strategic cunning” mentioned earlier, but mostly other things. (The conquistadors were disunited, but presumably were united in the ways that mattered.) At any rate, I expect AIs to be pretty good at coordinating too; they should be able to conquer the world just fine even while competing fiercely with each other. For more on this idea, see this comment.
By Daniel Kokotajlo
Acknowledgements
Thanks to Katja Grace for feedback on a draft. All mistakes are my own, and should be pointed out in the comments. Edit: Also, when I wrote this post I had forgotten that the basic idea for it probably came from this comment by JoshuaFox.