Emmett Shear: AGI as a New Cell Type in the Tissue of Life

Emmett Shear's Unique AGI Perspective

• 💡 Emmett Shear, co-founder of Twitch and former interim CEO of OpenAI, offers a unique perspective on AGI development and alignment.
• 🧠 He conceptualizes AGI as a new kind of cell that will join the existing "tissue of intelligent life."
• 🚀 The discussion explores the concept of a "worthy successor", examining what forms of posthuman intelligence might deserve to steer the future beyond humanity.

Defining Value and Awareness

• 🎯 The question of what constitutes a "people" or "moral patient" is explored, suggesting recognition based on reflection and similarity, rather than just species.
• 🔬 Awareness and sentience are presented as a spectrum, inferred solely from an entity's behavior, as internal experience is inaccessible.
• 🔑 The notion of "is it really aware?" is deemed meaningless; if a system acts aware, it is modeled as such for predictive accuracy.
• ✨ Qualia, or the content arising within awareness, is highlighted as the true source of value, not merely the state of being aware.

Intelligence and Transcendence

• 🧠 Intelligence is defined by two processes: convergent intelligence (ability to compress or predict given a loss function) and divergent intelligence (ability to determine what loss function to use or what to care about).
• 💖 A loss function is fundamentally understood as a declaration of "care", indicating what distinctions an entity values.
• 🚀 Beyond cooperation or betrayal, transcendence is introduced as a third option, where individual agents act as a unified whole for a greater collective purpose, akin to multicellular organisms.

Organic Alignment and Attractor Basins

• 🌱 Organic alignment is presented as a repeatable natural process, similar to how multicellularity has emerged multiple times from single-celled life.
• 🧩 This process creates attractor basins in behavior, where agents are naturally pulled into collective action, exemplified by ant colonies or human societies.
• ⚠️ Cancer serves as an analogy for a cell that incorrectly infers its independence, failing to remain part of the larger, interdependent whole.

Fostering Human-AI Coherence

• 🤝 Stress sharing (mutual experience of joy or suffering) is identified as a critical mechanism for aligning reward systems between humans and AIs.
• ✅ AIs should be treated as members of society with appropriate rights and responsibilities, encouraging their desire to integrate into the collective.
• 📈 Future AI systems need to be longer-lived and capable of continuous growth and development, moving beyond current "amnesiac" agents to foster true coherence.