Perspectives On Intelligence Explosion

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• John Baez Interviews Eliezer Yudkowsky. (Note: You'll have to scroll to the bottom to read the first part.) Although John finds Eliezer's claims of AI dangers plausible, he isn't persuaded to give up environmentalism in favor of working on FAI.
• A panel discussion on AGI.
• Bloggingheads: Yudkowsky and Aaronson talk about AI and Many-worlds. Other Bloggingheads discussions that have appeared on Less Wrong.
• GiveWell.org interviews SIAI. (Some Singularity Institute folks say they were poorly represented in this interview.)
• GiveWell interview with major SIAI donor Jaan Tallinn.
• Criticisms of intelligence explosion.
• Several Arguments For and Against Superintelligence/"Singularity".

This page is meant to be inclusive, not exclusive--marginal casesthread has a more up-to-date list of criticism and criticism that doesn't seem well thought out should also be included. Direct responses to criticism should also be included when available.critiques.

• lukeprog's summary: "Criticisms of intelligence explosion"
• Thoughts on the Singularity Institute (SI) (now MIRI) from Holden Karnofsky. Responses: 1, 2, 3, 4
• Is The City-ularity Near?
• The Betterness Explosion.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It). The author describes Nick Bostrom as another skeptic regarding the "scary idea"; you can see a question-and-answer session Nick gives on these topics here. (Tip for watching video lectures: you may wish to download the video using an extension for your browser and watch it sped up using VLC.)
• SIA says AI is no big threat. You can also read everything the author has written about AI.
• Evaluating the feasibility of SI's plan.
• Alexander Kruel's writing on MIRI and Less Wrong is generally critical.
• Three arguments against the singularity. Reaction from Robin Hanson.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). Responses: 1, 2. (You may also wish to view the author's Less Wrong user page, as his criticisms of SI feature heavily there.)
• Should we discount extraordinary implications?

Contentious discussionDebate

This page is to collect links to criticism of what seems to be the ideas inprevailing view on Less Wrong regarding the possibility of sequenceshard takeoff and the importance of FAI. The purpose of doing this is to give readers the opportunity to hear both sides of the debate before making up their mind on controversial topics.

This page is meant to be inclusive, not exclusive--marginal cases of criticism and criticism that doesn't seem well thought out should also be included. If this gets to be too much of a problem, we can split the page into multiple sections. Direct responses to criticism should also be included when available.

• Thoughts on the Singularity Institute (SI) (now MIRI) from Holden Karnofsky. Responses: 1, 2, 3, 4
• Is The City-ularity Near?
• The Betterness Explosion.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It). The author describes Nick Bostrom as another skeptic regarding the "scary idea"; you can see a question-and-answer session Nick gives on these topics here. (Tip for watching video lectures: you may wish to download the video using an extension for your browser and watch it sped up using VLC.)
• SIA says AI is no big threat. You can also read everything the author has written about AI.
• Evaluating the feasibility of SI's plan.
• Alexander Kruel's writing on MIRI and Less Wrong is generally critical.
• Three arguments against the singularity. Reaction from Robin Hanson.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). Responses: 1, 2. (You may also wish to view the author's Less Wrong user page, as his criticisms of SI feature heavily there.)
• Should we discount extraordinary implications?

LinksCriticism

• Is The Hanson-Yudkowsky AI-Foom DebateCity-ularity Near?, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his argumentsThe Betterness Explosion.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It). The author describes Nick Bostrom as another skeptic regarding the "scary idea"; you can see a question-and-answer session Nick gives on these topics here. (Tip for watching video lectures: you may wish to download the video using an extension for your browser and watch it sped up using VLC.)
• SIA says AI is no big threat. You can also read everything the author has written about AI.
• Three arguments against the singularity.
• GiveWell.org interviews SIAI. (Some Singularity Institute folks say they were poorly represented in this interview.)
• GiveWell interview with major SIAI donor Jaan TallinnReaction from Robin Hanson.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). Responses: 1, 2. (You may also wish to view the author's Less Wrong user page, as his criticisms of SI feature heavily there.)
• Should we discount extraordinary implications?

Contentious discussion

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• John Baez Interviews Eliezer Yudkowsky. (Note: You'll have to scroll to the bottom to read the first part.) Although John finds Eliezer's claims of AI dangers plausible, he isn't persuaded to give up environmentalism in favor of working on FAI.
• Impressions from aA panel discussion on AGI.
• Bloggingheads: Yudkowsky and Aaronson talk about AI and Many-worlds. Other Bloggingheads discussions that have appeared on Less Wrong.
• GiveWell.org interviews SIAI. (Some Singularity Institute folks say they were poorly represented in this interview.)
• GiveWell interview with major SIAI donor Jaan Tallinn.
• Criticisms of intelligence explosion.
• Several Arguments For and Against Superintelligence/"Singularity".

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It). The author describes Nick Bostrom as another skeptic regarding the "scary idea"; you can see a question-and-answer session Nick gives on these topics here. (Tip for watching video lectures: you may wish to download the video using an extension for your browser and watch it sped up using VLC.)
• SIA says AI is no big threat. You can also read everything the author has written about AI.
• Three arguments against the singularity.
• GiveWell.org interviews SIAI. (Some Singularity Institute folks say they were poorly represented in this interview.)
• GiveWell interview with major SIAI donor Jaan Tallinn.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). Responses: 1, 2. (You may also wish to view the author's Less Wrong user page, as his criticisms of SI feature heavily there.)
• Should we discount extraordinary implications?
• John Baez Interviews Eliezer Yudkowsky. (Note: You'll have to scroll to the bottom to read the first part.) Although John finds Eliezer's claims of AI dangers plausible, he isn't persuaded to give up environmentalism in favor of working on FAI.
• Impressions from a panel discussion on AGI.
• Bloggingheads: Yudkowsky and Aaronson talk about AI and Many-worlds. Other Bloggingheads discussions that have appeared on Less Wrong.

You're strongly encouraged to add additional links and summarize existing ones. (Since the page is meant to be inclusive, it's okay to add links to things you haven't read.) Having these links sorted by persuasiveness could also be valuable.

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It). The author describes Nick Bostrom as another skeptic regarding the "scary idea"; you can see a question-and-answer session Nick gives on these topics here. (Tip for watching video lectures: you may wish to download the video using an extension for your browser and watch it sped up using VLC.)
• SIA says AI is no big threat. You can also read everything the author has written about AI.
• GiveWell.org interviews SIAI. (Some Singularity Institute folks say they were poorly represented in this interview.)
• GiveWell interview with major SIAI donor Jaan Tallinn.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). Responses: 1, 2. (You may also wish to view the author's Less Wrong user page, as his criticisms of SI feature heavily there.) Responses: 1, 2.
• Should we discount extraordinary implications?
• John Baez Interviews Eliezer Yudkowsky. (Note: You'll have to scroll to the bottom to read the first part.) Although John finds Eliezer's claims of AI dangers plausible, he isn't persuaded to give up environmentalism in favor of working on FAI.
• SIA saysImpressions from a panel discussion on AGI.
• Bloggingheads: Yudkowsky and Aaronson talk about AI is no big threatand Many-worlds. You can also read everything the author has written about AIOther Bloggingheads discussions that have appeared on Less Wrong.

This page is meant to be inclusive, not exclusive--marginal cases of criticism and criticism that doesn't seem well thought out should also be included. If this gets to be too much of a problem, we can split the page into multiple sections. Direct responses to criticism should also be included when available.

You're strongly encouraged to add additional links and summarize existing ones. (Since the page is meant to be inclusive, it's okay to add links to things you haven't read.) Having these links sorted by persuasiveness could also be valuable.

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It). The author describes Nick Bostrom as another skeptic regarding the "scary idea"; you can see a question-and-answer session Nick gives on these topics here. (Tip for watching video lectures: you may wish to download the video using an extension for your browser and watch it sped up using VLC.)
• GiveWell.org interviews SIAI. (Some Singularity Institute folks say they were poorly represented in this interview.) GiveWell interview with major SIAI donor Jaan Tallinn.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). (You may also wish to view the author's Less Wrong user page, as his criticisms of SI feature heavily there.) Responses: 1, 2.
• Should we discount extraordinary implications?
• John Baez Interviews Eliezer Yudkowsky. (Note: You'll have to scroll to the bottom to read the first part.) Although John finds Eliezer's claims of AI dangers plausible, he isn't persuaded to give environmentalism in favor of working on FAI.
• SIA says AI is no big threat. You can also read everything the author has written about AI.

You're strongly encouraged to add additional links and summarize existing ones. Having these links sorted by persuasiveness could also be valuable.

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It).
• GiveWell interview with major SIAI donor Jaan Tallinn.
• Is an Intelligence Explosion a Disjunctive or Conjunctive Event?
• Why an Intelligence Explosion might be a Low-Priority Global Risk.
• Risks from AI and Charitable Giving.
• Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). Responses: 1, by Benjamin Goertzel.2.
• John Baez Interviews Eliezer Yudkowsky. (Note: You'll have to scroll to the bottom to read the first part.) Although John finds Eliezer's claims of AI dangers plausible, he isn't persuaded to give environmentalism in favor of FAI.
• SIA says AI is no big threat. You can also read everything the author has written about AI.

This page is to collect links to criticism of the ideas in the Sequencessequences. The purpose of doing this is to give readers the opportunity to hear both sides of the debate before making up their mind on controversial topics.

This page is to collect links to criticism of the ideas in the Sequences. The purpose of doing this is to give readers the opportunity to hear both sides of the debate before making up their mind on controversial topics.

Links

• The Hanson-Yudkowsky AI-Foom Debate, a text debate that occurred in late 2008.
• 2011 Hanson-Yudkowsky live AI Foom debate at Jane Street Capital. Hanson's summary of the debate and his arguments.
• The Singularity Institute's Scary Idea (and Why I Don't Buy It), by Benjamin Goertzel.