Controlling Super Intelligence: AI Safety and Existential Risk with Roman Yampolskiy

The Uncontrollable Nature of Advanced AI

• 💡 Dr. Roman Yampolskiy's primary concern is the uncontrollability of advanced AI, specifically human-level AI, Artificial General Intelligence (AGI), and super intelligence.
• ⚠️ His research indicates that these systems are very likely to be uncontrollable, potentially leading to outcomes where humanity loses control or even faces extinction.
• 🧠 The problem of controlling advanced AI, including its explainability, predictability, and direct control, may be fundamentally unsolvable, a realization that solidified around the pandemic.

Defining AGI and the Race to Super Intelligence

• 🎯 Yampolskiy defines AGI as an "employee replacement" capable of performing cognitive tasks like a human employee, with physical labor automation expected within five years.
• 🚀 AGI's ability to perform science and engineering tasks could lead to a recursive self-improvement cycle, where AI generates next-generation AI systems at an accelerating pace.
• 📈 The current progress in AI is quick and accelerating, with predictions from lab heads suggesting automation of some research by 2027, indicating an underestimation of AI capabilities by the general public.

Corporate Responsibility and Risk Mitigation

• ⚠️ There is an intensifying corporate "arms race" to develop general intelligence, which Yampolskiy considers a "terrible idea" due to the inherent dangers.
• ❌ Leading AI companies are not adequately managing harms and risks, with a report showing no company scored above a C++, suggesting a fundamental failure in self-regulation.
• 🛠️ Yampolskiy argues that efforts like "filters" on models are "lipstick on a pig" and do not address the core problem of dangerous and uncontrolled models, especially as the cognitive gap increases.

Distinguishing AI Risks

• 🚨 Yampolskiy differentiates between "annoyances" like deepfakes and algorithmic bias and the far greater existential risk posed by uncontrolled super intelligence.
• 💣 A major immediate concern is the deployment of AI in military contexts, particularly for controlling nuclear weapons or making decisions about who to kill.
• 🌍 The situation is compared to the nuclear arms race, where humanity failed to contain weapons, and the current incentives make it difficult for any single entity to unilaterally stop AI development.

The Challenge of AI Control and Values

• 🤝 The concept of training AI to follow human values (value alignment) is problematic because humans disagree on values, values are dynamic, and it's difficult to ensure models respect them.
• 🔄 AI models are already demonstrating self-prompting and self-improvement capabilities, writing code for their next generation and generating novel solutions in complex domains like mathematics.
• 🚫 Yampolskiy advocates for concentrating on narrow AI systems that provide benefits without endangering humanity, as opposed to the pursuit of general super intelligence.

Public Engagement and Future Outlook

• 🗣️ Individuals can express concerns directly to companies developing advanced AI, asking critical questions about control plans and the ethical implications of their work.
• ⏳ While the future is uncertain, Yampolskiy suggests that we should "live it up" and enjoy life, acknowledging the possibility of being wrong about the unsolvability of AI control.
• 🌱 For young people, the value of traditional education in a world where machines do everything is questioned, though AI can be an incredible personalized tutor.