Max Tegmark on Halting Superintelligence Development

Call for Superintelligence Prohibition

• 💡 A statement calls for a prohibition on the development of superintelligence until broad scientific consensus on safety and controllability is reached, alongside strong public buy-in.
• 🎯 Signatories include diverse leaders like Steve Wozniak, Prince Harry, and faith leaders, united by the common concern for humanity's future.
• 🧠 Leading AI scientists, described as the "Einstein and Oppenheimer of today," have also signed, acknowledging the risks inherent in their own work.

The Dangers of Unchecked AI Ambition

• 🚀 Figures like Sam Altman are perceived as aiming to "meet God" through AI creation, with some advocating for humans to "merge with machines."
• ⚠️ This ambition is seen as an assault on democracy, with a few individuals in Silicon Valley making decisions for all of humanity regarding human evolution and AI integration.
• 👨‍👩‍👧‍👦 The idea of forcing children to merge with machines without consent is highlighted as a deeply concerning prospect.

Countering the "Unstoppable" Narrative

• 🗣️ The argument that AI development is unstoppable is a "stop trick" used by companies to discourage resistance, similar to how an invading army might claim invincibility.
• 🚫 Historically, society has chosen to prohibit dangerous technologies, such as human cloning, despite potential financial gains, demonstrating that progress can be controlled.
• 🇨🇳 The example of China punishing a scientist for human cloning illustrates that governments, even those focused on control, can halt risky advancements.

Two Races: Dominance vs. Existential Risk

• 🇺🇸🇨🇳 There are two distinct races between the US and China: one for economic and military dominance using controllable tools, and another for building uncontrollable superintelligence, which is a "suicide race."
• 🧊 Analogous to the Cold War, where both superpowers avoided a nuclear suicide race, nations can choose not to engage in the race towards uncontrollable AI.
• 🤝 The goal is not necessarily trust between nations, but a mutual understanding that pursuing uncontrollable superintelligence leads to mutual destruction.

Establishing Safety Standards for AI

• ⚖️ Just as biotech and restaurants are subject to strict safety regulations (e.g., FDA, health inspectors), AI companies should not be exempt from safety standards.
• 💊 The tragedy of thalidomide, which caused severe birth defects, led to the creation of regulatory bodies like the FDA, demonstrating the necessity of oversight.
• 💰 Exempting AI companies from safety standards is akin to providing corporate welfare, allowing them to operate without accountability.

The Purpose of the Superintelligence Statement

• 📢 The statement aims to validate concerns about AI, showing that individuals with these worries are not alone and have broad support.
• 🗣️ It encourages people to speak up against the risks of AI without fear of being labeled as alarmists.
• 📊 Polls indicate that a vast majority of Americans do not want a race to superintelligence, yet it is being pushed forward.
• 🤝 The statement seeks to foster a "Team Human" approach, uniting people to address the existential threat posed by AI.
• 🌐 The initiative is promoted through websites like futureoflife.org and superintelligencestatement.org, encouraging public support and signatures.