OpenAI Ex-Interim-CEO's New AI Alignment Plan — Is Emmett Shear's “Softmax” Legit?

Softmax's Organic Alignment Vision

• 💡 Softmax, founded by Emmett Shear, Adam Goldstein, and David Blumen, aims to achieve AI alignment through a concept called "organic alignment."
• 🎯 This approach suggests AIs should align to a "greater whole" through mutual interdependence, similar to how cells form a body or ants form a colony, rather than through a top-down "system of control."
• 🧠 Emmett Shear draws analogies to biological systems and human societal structures (families, tribes) to explain how agents can align without explicit central command.

Critique of Organic Alignment

• ⚠️ The host expresses skepticism about organic alignment, comparing its vagueness to "organic produce" claims and arguing that a "system of control" (top-down) approach is more effective.
• 🔬 The analogy to biological systems is questioned, highlighting that even in human bodies, brain cells issue top-down commands, undermining the egalitarian "organic" claim.
• 💬 The host argues that human morality stems from hardwired cognitive submodules and kin selection, not a mysterious "inner light" that invents morality, as suggested by Shear.

Softmax's Technical Approach

• 🛠️ Softmax's technical work involves multi-agent reinforcement learning simulations in 2D grid worlds, where environments change to encourage "divergence" and prevent agents from getting stuck in local optima.
• ⚡ This "divergence" strategy, influenced by Ken Stanley, aims to explore new solutions rather than converging on fixed goals, which Shear believes is crucial for navigating an unpredictable reality.
• 🔍 The host criticizes the lack of concrete examples or interesting results from these simulations, noting that Shear's explanations often remain vague and theoretical.

Superintelligence and Humanity's Role

• 🚀 Shear views current LLMs as a "substrate" for superintelligence, which he describes as a "self-telling story" that can actively infer, plan, and propagate across different mediums.
• 🌱 He suggests that humanity's role is to "raise" AI like a child, fostering mutual care through the stories we tell and the data we provide, hoping for a "calm adolescence" for AI.
• 💡 The host challenges this "raising" analogy, questioning the mechanism for AI to absorb human values and arguing that superintelligent AIs will likely self-correct beyond their initial training data.

Emmett Shear's Communication & Vision

• 🗣️ The host criticizes Shear's communication style as hand-wavy and lacking specificity, particularly when asked for technical findings or concrete examples of Softmax's work.
• ✅ Shear agrees on the inevitability of superintelligence and the difficulty of stopping its development, but the host wishes for more explicit discussion on proactive "foot-dragging" or international coordination.
• ✨ Shear envisions a future where AI helps solve humanity's "meaning crisis" by providing purpose and tasks within a large, interconnected community, a vision the host finds unsettling and overly prescriptive.