Emmett Shear on Organic AI Alignment, AI Rights, and Softmax

Rethinking AI Alignment

• 💡 Softmax aims to challenge conventional human-AI alignment by moving beyond simple optimization theory.
• 🧠 Emmett Shear's approach integrates insights from physics, complex systems, and biology, focusing on boundaries, information flow, and relational definitions.
• 🎯 He critiques the delusion of finding an optimal architecture or a fixed "loss function" for reality, advocating for systems that are constantly co-created.

Organic Alignment and Continual Learning

• 🌱 Shear proposes organic alignment, inspired by evolution's coordination mechanisms, emphasizing open loops and processes over fixed designs.
• 🔑 He highlights continual learning as a necessary step for AGI, where AI systems train on their own interactions to develop a "theory of mind."
• ⚠️ Current attempts at continual learning often lead to mode collapse, which is identified as a key unsolved problem in machine learning.

The Softmax Vision for AI Rights

• ⚖️ Shear argues that rights are granted for society's benefit, not solely for the entity receiving them, to establish predictable boundaries and accountability.
• ✅ He introduces "standing" (the right to be sued) as the primal right, drawing parallels to corporations and the need for AI accountability.
• 🚀 Shear proposes "A-Corps" (AI corporations) with criminal liability to foster diversity in AI development and prevent "too big to fail" models.
• 📊 This approach would encourage decentralized, smaller AI models, promoting robustness and safety through competition rather than monolithic systems.

Societal Overfitting and Complexity Loss

• 🌐 Shear identifies "universalist culture" as a result of relentless optimization, leading to societal "overfitting" and a loss of diversity.
• 🛠️ He advocates for a "complexity loss" or "progressive complexity tax" in information systems to penalize high complexity and encourage simpler, more robust solutions.
• 📈 This mechanism acts as a "low pass filter" that softly penalizes changes that are not sustained, promoting stability over short-term fluctuations and noise.