Note 1 : This was posted to EA Forum on May 31 here, as a submission for the 2023 Open Philanthropy AI Worldviews contest, due May 31, 2023. It addresses Question 1: “What is the probability that AGI is developed by January 1, 2043?”

Note 2: I'm developing a series of several more detailed essays on active moral stigmatization of AI as a strategy for pausing, slowing, or stopping dangerous AI development; so I'd welcome any constructive feedback on this initial essay.

Overview

People tend to view harmful things as evil, and treat them as evil, to minimize their spread and impact. If enough people are hurt, betrayed, or outraged by AI applications, or lose their jobs, professional identity, and sense of purpose to AI, and/or become concerned about the existential risks of AI, then an intense public anti-AI backlash is likely to develop. That backlash could become a global, sustained, coordinated movement that morally stigmatizes AI researchers, AI companies, and AI funding sources. If that happens, then AGI is much less likely to develop by the year 2043. Negative public sentiment could be much more powerful in slowing AI than even the most draconian global regulations or formal moratorium, yet it is a neglected factor in most current AI timelines.

Introduction

The likelihood of AGI being developed by 2043 depends on two main factors: (1) how technically difficult it will be for AI researchers to make progress on AGI, and (2) how many resources – in terms of talent, funding, hardware, software, training data, etc. – are available for making that progress. Many experts’ ‘AI timelines’ for predicting AI development assume that AGI likelihood will be dominated by the first factor (technical difficulty), and assume that the second factor (available resources) will continue increasing.

In this essay I disagree with that assumption. The resources allocated into AI research, development, and deployment may be much more vulnerable to public outrage and anti-AI hatred than the current AI hype cycle suggests. Specifically, I argue that ongoing AI developments are likely to provoke a moral backlash against AI that will choke off many of the key resources for making further AI progress. This public backlash could deploy the ancient psychology of moral stigmatization against our most advanced information technologies. The backlash is likely to be global, sustained, passionate, and well-organized. It may start with grass-roots concerns among a few expert ‘AI doomers’, and among journalists concerned about narrow AI risks, but it is likely to become better-organized over time as anti-AI activists join together to fight an emerging existential threat to our species. (Note that this question of anti-AI backlash likelihood is largely orthogonal to the issues of whether AGI is possible, and whether AI alignment is possible.)

I’m not talking about a violent Butlerian Jihad. In the social media era, violence in the service of a social cause is almost always counter-productive, because it undermines the moral superiority and virtue-signaling strategies of righteous activists. (Indeed, a lot of ‘violence by activists’ turns out to be false flag operations funded by vested interests to discredit the activists that are fighting those vested interests.)

Rather, I’m talking about a non-violent anti-AI movement at the social, cultural, political, and economic levels. For such a movement to slow down the development of AGI by 2043 (relative to the current expectations of Open Philanthropy panelists judging this essay competition), it only has to arise sometime in the next 20 years, and to gather enough public, media, political, and/or investor support that it can handicap the AI industry’s progress towards AGI, in ways that have not yet been incorporated into most experts’ AI timelines.

An anti-AI backlash could include political, religious, ideological, and ethical objections to AI, sparked by vivid, outrageous, newsworthy failures of narrow AI systems. An anti-AI backlash could weakly delay AI research through government regulation. But it could strongly delay AI research through socio-cultural dynamics such as AI research becoming morally taboo, socially stigmatized, religiously condemned, and/or politically polarized. For example, if being an AI researcher became as publicly stigmatized as being a white nationalist, a eugenicist, a sexist, or a transphobe, then AI research would be largely abandoned by any researchers sensitive to social pressure, and AGI would not be developed for a long time.

Thus, we can invert the question of AGI timelines, and consider the possible timelines for an anti-AI backlash. Rather than asking ‘What is the likelihood that we’ll have AGI by 2043?’, we could ask ‘What is the likelihood that we will see an anti-AI backlash by 2043 – a backlash that is strong enough to slow down AGI development?’ I’d argue that the answer to this second question is fairly high. Even just in the last few weeks (as of May 31, 2023), we’ve seen a dramatic increase in public attention on AI risk, public and government concern about AI, and the beginnings of an anti-AI backlash on social media, such as Twitter.

(Note that in this essay I’m not taking a position on whether an anti-AI backlash would be a good thing or a bad thing; I’m just doing a preliminary analysis of how such a backlash could slow down AGI timelines.)

Triggers for an anti-AI backlash

The general public is already culturally primed for an anti-AI backlash. Ever since the novel Frankenstein (1818), we’ve had generations of science fiction novels, movies, TV shows, computer games, and other media portraying the dangers of creating artificial intelligence. Most living people in developed countries have been exposed to these cautionary tales. They’ve mostly seen 2001: A Space Odyssey, The Terminator, Ex Machina, Black Mirror, and Westworld. They’re often the first things that ordinary people think about when they think about AI. And most adults have first-hand experience of playing computer games against powerful (but narrow) AI, e.g. trying to win ‘Civilization’ on ‘god mode’ difficult level.

The triggers for an anti-AI backlash don’t need to create moral stigma from scratch. They just need to connect these latent cultural fears of AI to current real-world AI issues. I’ll call these issues ‘triggers’, and there are several kinds that seem quite likely to provoke moral stigmatization of AI within the next 20 years.

Trigger 1: Unemployment

People get pretty upset when they lose their jobs. The closer we get to AGI, the more job losses we’ll see. And, for any ‘new jobs’ that open up due to increased economic activity, AI systems will probably be able to learn the new job faster than humans will be able to re-train to do them.

Insofar as Large Language Models are making faster progress in human-style information processing than autonomous robotics are making in doing physical tasks, AI job losses may start hitting white-collar professional who do ‘brain work’ before they hit blue-collar workers doing physical work. These white-collar professionals may include millions of suddenly unemployed lawyers, accountants, journalists, teachers, academics, medical staff, pharmacists, software engineers, graphic designers, architects, and civil engineers.

Such people are typically highly educated, politically engaged, and prone to adopting new moral stigmas through social media. If they’re unemployed, they would have all the time in the world to organize an anti-AI backlash movement. If they have some real estate equity, investment assets, and credit, they may have the money to keep fighting for a while, even without an income. If they have kids, who face poor career prospects in turn due to ongoing AI developments, they may feel the righteous fury of parents who are motivated to do anything necessary to secure a viable future for their next generation. Thus, AI-imposed unemployment is likely to provoke an anti-AI backlash, probably in the time scale of 5-20 years from now.

Trigger 2: Sex

Moral stigmatization often focuses on human sexuality. Sexual practices outside the mainstream have often provoked furious moral condemnation, across cultures and across history – whether it’s incest, polygamy, prostitution, cheating, BDSM, polyamory, or porn. As narrow AI gets applied to goods and services related to human sexuality, there are likely to be all kinds of moral backlashes from diverse groups, ranging from Christian conservatives to woke feminists.

New information technologies are often applied first to create new sexual content. Internet Rule 34 says ‘If it exists, there is porn of it; no exceptions’. A variant will be ‘If AI can make porn of it, there will be porn of it’. Possible applications of AI in the sexual domain have focused on AI-generated porn and erotica (whether photos, audio, video, or stories), deepfake porn, interactive girlfriends and boyfriends, and sexbots.

A key trigger for an anti-AI backlash could be the moral outrage and sexual disgust provoked by sexual applications of narrow AI, such as highly habit-forming interactive VR porn, or customized erotic chatbots with the voices, mannerisms, ad personality traits of someone’s neighbors, co-workers, or ex-lovers.

The most salient, intimate, and controversial application of AI in the next couple of decades will be, essentially, the production of interactive sex slaves – whether in real physical bodies, VR avatars, 2-D deepfake porn, or auditory chatbots. The moral condemnation of slavery remains very strong – it just hasn’t been applied yet to digital slaves. When AI researchers start to be seen as breeders and traders of digital sex slaves, they’re likely to be strongly stigmatized.

Many of these sexual AI applications will take highly controversial forms. Pedophiles will buy AI sexbots with children’s bodies. Sadistic psychopaths will use disposable AI sexbots that can be flogged, cut, and branded, and that scream in realistic pain. Guys who like futanari porn will use sexbots that combine the primary and secondary sexual traits of males and females. AI-generated deepfake porn of politicians, tech billionaires, media celebrities, journalists, and activists is especially likely to provoke the wrath of the rich, powerful, and influential.

The marketing and use of these sexual AI applications may be private at first, but there will inevitably by news coverage, and it will be written to provoke maximum moralistic outrage, because moralistic outrage sells, and gets clicks, and gets shares on social media.

Trigger 3: Violence

Many AI researchers have signed pledges not to develop lethal autonomous weapons (LAWs), such as ‘slaughterbots’. However, there are many other applications of narrow AI that could lead to widespread dangers, injuries, and deaths. Such violence often provokes moral outrage and intense stigmatization of the technologies involved.

The big danger here is not so much that AI safety engineers will stupidly overlook some obviously dangerous failure mode in their systems. Rather, the danger is that rogue nation-states, terrorists, bad actors, resentful former employees, aggrieved nihilists, creepy stalkers, or mischievous youth will manipulate or hack the AI systems to cause targeted deaths of mass carnage. Bad actors could hack self-driving cars to cause huge pile-ups on highways that lead to dozens of deaths. AI drones could be modified by terrorists, criminal gangs, or violent activists to cause mass shootings or explosions at public events. Autonomous assassination drones with face-recognition abilities and long-term loitering abilities could kill major heads of state. Obsessive stalkers could use AI systems to track, harass, and harm their sexual victims. Anarchists, anti-capitalists, and eco-activists who hate resource-intensive industries could hack AI factory control systems to cause horrific industrial accidents. Religious extremists could use AI propaganda systems to promote religious radicalization, terrorism, and warfare.

All of these violent AI applications will, of course, be dismissed and disavowed by the AI industry. But the public may notice the common denominator: AI allows highly effective, targeted violence that is displaced in time and space from the humans directing the violence. This increases the effectiveness and decreases the risks of doing all kinds of mayhem. This will strike many ordinary people as horrifying and outrageous, and will reinforce anti-AI sentiment.

Other triggers

Apart from unemployment, sex, and violence, there are many other applications of narrow AI that could exacerbate an anti-AI backlash. These include harmful effects on AI on women, children, elders, racial minorities, and sexual minorities. These include harmful effects of AI propaganda in political polarization and religious intolerance. Biomedical AI systems for drug discovery could lead to new, highly addictive, psychosis-inducing recreational drugs rather than cures for cancer. AI applied to consumer advertising, gambling, and investments could lead people into over-spending, debt, bankruptcy, divorce, and ruin. The number of harmful things that could go wrong with narrow AI systems is almost limitless – but each new type of harm will be an occasion for sensationalist news coverage, public outrage, virtue signaling, political condemnation, and moral stigmatization of AI.

AI chokepoints that could delay AGI

So what if there’s an anti-AI backlash? What could ordinary people actually do to slow down AI research, given the arms race dynamics between AI companies (such as Microsoft vs. Google) and nation-states (such as the US vs. China)? This section addresses some key resources required for AGI development that could be choked off by an anti-AI backlash. It’s not an exhaustive list of the ways that moral stigmatization of AI could handicap AI research. It’s just intended to give a sense of how strongly and comprehensively an anti-AI backlash could lead to another ‘AI winter’, or even to a decades-long ‘AI ice age’.

Chokepoint 1: AI Talent

If AI research becomes strongly morally stigmatized, all the prestige and coolness of being an AI researcher would evaporate. Moral stigmatization of a career does not just mean the career suffers a slight decline in status relative to other careers. No. The psychology of moral stigmatization means the general public views the career as evil, and views the people working in the career as morally tainted by that evil. Intense moral stigma against AI would mean that being an AI researcher is seen as being about as reputable as being a convicted sex offender, a Nazi racist, an arms dealer, or a mass murderer. The public would view AI researchers as hubris-driven mad scientists with psychopathic traits and genocidal aspirations.

Moral stigmatization of AI research would render AI researchers undateable as mates, repulsive as friends, and shameful to family members. Parents would disown adult kids involved in AI. Siblings wouldn’t return their calls. Spouses would divorce them. Landlords wouldn’t rent to them.

Once the anti-AI backlash renders AI researchers socially, sexually, and professionally toxic, this would radically reduce the quantity and quality of talent working in AI. People with the technical skills to do AI research would exit the field, and would work instead on cybersecurity, or crypto, or non-AI software, or robotics, or whatever. They would have many career options that aren’t viewed as evil by lots of people they meet.

People in some fields have already developed pretty thick skins for resisting stigma. Researchers in controversial areas such as behavior genetics, evolutionary psychology, intelligence research, sex differences, and race differences have been subject for decades to moral stigmatization, hostile stereotyping, career handicaps, lack of funding, and attacks by journalists. They’ve become self-selected for orneriness, disagreeableness, intellectual courage, emotional stability, and self-sufficiency, and they’re learned many coping strategies. By contrast, AI researchers have little experience of being morally stigmatized. They’re used to high status, prestige, income, and coolness. They may be shocked when the public suddenly turns against them and paints them as evil mad scientists consumed by hubris and misanthropy. In other words, the pool of AI talent is highly vulnerable to stigmatization, and has few defenses against it. Faced with a choice between staying in a highly stigmatized field (AI) versus switching to another highly-paid, intellectually engaging computer science field that is not highly stigmatized (e.g. gaming, cybersecurity, crypto), most AI researchers may jump ship and leave AI.

Chokepoint 2: AI Funding

A strong enough anti-AI backlash would lead to AI funding drying up. Investors have become quite sensitive to ‘ESG criteria’ concerning environmental, social, and governance issues. If AI becomes morally stigmatized, ESG criteria could quickly and easily include AI as a disqualifying taboo. Any company involved in AI would receive low ESG scores, and would attract less ‘ethical investment’.

Apart from formal ESG criteria, individual and institutional investors tend to avoid companies widely perceived as reckless, evil, and inhumane. Many investors already avoid companies involved in weapons, alcohol, tobacco, porn, or gambling. If AI becomes seen as a horrifying new weapon, an addictive entertainment, and/or an insanely risky species-level gamble, it would combine all the worst evils of these already-stigmatized industries.

Investors may aspire to be rational maximizers of risk-adjusted returns. But investors are also social primates, subject to the same social and moral pressures that shape human behavior in every other domain of life. High Net Worth Individuals (‘rich people’) often set up family offices to handle their investments, assets, and trusts for their kids and grand-kids. These family offices are specifically designed to take a long-termist, multi-generational perspective on the preservation and enhancement of dynastic wealth and power. That long-termist perspective naturally leads to a concern about multi-decade technological changes, geopolitical risks, global catastrophic risks, and existential risks. If AI becomes morally stigmatized as a major existential risk, family offices and their investment professionals will not want to deploy their capital in AI companies that could lead the rich people’s kids and grand-kids not to die out before the end of the 21st century.

The investment world, like every human world, is prone to moral fads and fashions. Some companies and industry sectors become viewed as morally righteous, saintly, and inspiring; others become viewed as morally disgusting, sinful, and degrading. The psychology of moral disgust runs on the logic of contagion: anything in, around, or near a morally stigmatized activity becomes morally stigmatized by proxy. This means that if a large publicly traded corporation such as Microsoft or Google happens to include a much smaller organization (such as OpenAI or DeepMind) that becomes stigmatized, the large corporation also becomes morally stigmatized. Fewer people want to invest in it. They don’t want their portfolio contaminated by the second-hand evil. As fewer investors are buying and more are selling, the share price falls. As the share price falls, other investors see the writing on the wall, and panic-sell. Hedge funds start aggressively shorting the stock. Soon the corporations face a dilemma: either they shut down or sell off the tainted AI organization poisoning their shareholder value from within, or they continue seeing their share price fall off a cliff – until they get acquired in a hostile takeover by new investors who are willing to cut the AI cancer out of the corporation, to save the rest of the company.

A few anti-ethical investors might see AI as a clever contrarian play, and might think AI company stocks are temptingly under-valued, and will become great investments after the moral stigma fades. But the stigma might not fade, and they may be left facing huge capital losses.

Chokepoint 3: Suppliers

Moral contagion flows out in all directions. If AI starts to be seen as evil, any other organization that does business with AI researchers or AI companies will be seen as evil, or at least evil-adjacent. They will be stigmatized by association, as often happens in ‘cancel culture’. AI research depends on all kinds of suppliers of goods and services, utilities, computational infrastructure, and business infrastructure.

A sort of ‘ethical back-propagation’ would happen, where the moral stigma of AI would propagate backwards along the supply chain, tainting every person and company that provides essential goods and services to AI research.

In response, every supplier who is sensitive to the anti-AI backlash may withdraw their support from AI research groups. This may include everyone supplying GPU hardware, software, cloud computing resources, office space, legal services, accounting services, banking services, and corporate recruiting services. AI businesses may find that no reputable lawyers, bookkeepers, banks, or headhunters are willing to work with them. At a more mundane level, AI groups may find that they cannot find reputable businesses willing to supply them with tech support, back-office staff, office temps, caterers, drivers, janitors, or security staff. If some companies are not willing to do business with cannabis shops, porn producers, drug gangs, arms dealers, racketeers, human traffickers, or other stigmatized forms of economic activity, and if AI becomes stigmatized to a similar level, AI research will be handicapped, and will slow down.

The supplier issue could also affect AI researchers in their personal lives. If AI is widely seen as a work of reckless, hubristic evil, AI researchers may find that landlords are not willing to rent to them, coop boards are not willing to let them buy condos, and daycare centers and private schools are not willing to care for their kids. Bodyguards and police may think they’re too disgusting to protect. Therapists may advise them to ‘seek help elsewhere’. They may even find spiritual services getting choked off, as their priest, pastor, or rabbi shun them for the sinful way they make a living.

Chokepoint 4: Laws and regulations

Informal moral stigmatization often leads to formal government regulations and laws governing new activities and technologies. Indeed, it’s often difficult to coordinate bipartisan support for new regulations and laws constraining something unless there is already a foundation of public stigmatization against that thing. Once the horrors of chemical weapons were witnessed in World War 1, and the public viewed mustard gas and other agents as morally outrageous, it was fairly easy to develop international bans on chemical weapons. Once human cloning became morally stigmatized in the 1990s, it was fairly easy to implement government bans and scientific norms against human cloning. Conversely, it’s quite difficult to sustain regulations and laws against something if the moral stigma against the thing erodes – as in the case of cannabis use gradually becoming destigmatized in the US since the 1960s, and legalization of recreational cannabis following in many states. Thus, moral stigma and government regulation often have mutually reinforcing functions.

In the case of AI, if an anti-AI backlash was sufficiently global in scale, and became a major focus of public concern in both the US and China, it may be much easier to develop international agreements to pause, constrain, or ban further AGI research. With global moral stigmatization of AI, global regulation of AI becomes feasible. Without global moral stigmatization of AI, global regulation of AI is probably impossible. Yet much of the work on AI governance seems to have ignored the role of informal moral stigmatization in creating, energizing, and sustaining formal international agreements.

If an anti-AI backlash gets formalized into strong laws and regulations against AGI development, leading governments could make it prohibitively difficult, costly, and risky to develop AGI. This doesn’t necessarily require a global totalitarian government panopticon monitoring all computer research. Instead, the moral stigmatization automatically imposes the panopticon. If most people in the world agree that AGI development is evil, they will be motivated to monitor their friends, family, colleagues, neighbors, and everybody else who might be involved in AI. They become the eyes and ears ensuring compliance. They can report evil-doers (AGI developers) to the relevant authorities – just as they would be motivated to report human traffickers or terrorists. And, unlike traffickers and terrorists, AI researchers are unlikely to have the capacity or willingness to use violence to deter whistle-blowers from whistle-blowing.

Laws and regulations by themselves would not be enough to significantly slow down AGI development. Bad actors would always be motivated to evade detection and accountability. However, it’s a lot harder to evade detection if there is a global moral stigma against AGI development, with strong public buy-in. From the public’s point of view, laws and regulations are simply ways to articulate, formalize, and implement moral stigmas that are already widely accepted in public discourse. In short, the public has already figured out what’s evil, and they just want government to use its monopoly on the legitimate use of force to deter and punish what’s evil. Thus, moral stigmatization super-charges the effectiveness of any formal laws and regulations around AI.

Often, if some activity becomes sufficiently stigmatized, regulators and law enforcement can apply existing laws in highly targeted ways to deter the activity. For example, laws against reckless endangerment and public endangerment could be applied to prosecute AGI research – if there was sufficient public and institutional belief that AGI imposes existential risks on citizens without their consent. The FBI could switch its focus from ‘white supremacy as the leading domestic terrorist threat’ to ‘AGI research as the leading domestic terrorist threat’ – and investigate and prosecute AI researchers accordingly. Note that government regulators and law enforcement agencies are often motivated to find and capitalize on any new threats that the public perceives. This provides pretexts for increasing their budgets, staff, and powers. If an anti-AI backlash becomes popular, many government workers will see this as a great opportunity to increase their status and power. Fighting against something widely considered an existential threat to humanity would sound like a pretty cool mission to a lot of FBI agents (in the US) or Ministry of State Security agents (in China). Thus, moral stigmatization of AI could lead quite quickly and directly to government investigations, audits, litigation, and prosecution of AI researchers and companies. Result: AGI development is slowed or stopped.

Conclusion

The social-psychological processes of moral stigmatization have evolved genetically and culturally over thousands of generations. Moral stigma plays crucial roles in solving group coordination problems, enforcing social norms, punishing anti-social behavior, and minimizing existential threats to groups. Stigmatization is both a deep human instinct and a powerful cultural tradition. It can solve problems that can’t be solved in any other way. This may include solving the problem of delaying AGI development until we have a better idea whether AI alignment is possible at all, and if it is possible, how to achieve it.