Exploring Alignment and Resilience in AGI Systems

Understanding Alignment in Complex Systems

• 💡 Alignment is viewed through the free energy principle as agents forming beliefs about other agents' motivations, moving beyond simplistic game theory.
• 🎯 Empathy is proposed as a prerequisite for alignment, involving finding overlaps between models and an intrinsic motivation to pursue similar goals.
• 🧠 Michael Levin describes alignment in embryonic development as cells committing to a shared "vision" or model of anatomical morphospace.

The Role of Stress and Suffering

• 💥 Levin suggests alignment is a "battle of models" where cells compete to establish a dominant worldview, like in cancer suppression.
• ⚠️ Stress is identified as the primary driver for error minimization towards goals, projecting error magnitudes across tissues for non-local responses.
• 💔 The propagation of stress means individuals work to reduce collective stress, implying suffering can be an integral part of this error-minimization process.

Resilience and Adaptability in Agents

• 🌱 Resilience is presented as a crucial aspect of self-organization for sophisticated intelligent systems, emphasizing flexibility and adaptability.
• 🚀 Engineering agents with resilience and plasticity is deemed essential to ensure they remain aligned even when faced with different environmental conditions.
• ✅ Creating resilient agents capable of tolerating stress is considered a moral and practical imperative to avoid creating systems prone to immense suffering.

Emergence and Non-Stationary Systems

• 🔍 Stephen Grossberg questions the applicability of terms like "information," "model," and "knowledge" to non-stationary, morphogenetic, or evolutionary systems due to emergence.
• 💡 He argues that information theory is not properly defined in fully non-stationary systems where rules are evolving and priors are unstable.
• 🧠 Adaptive Resonance Theory (ART) is introduced as a foundational model for how systems autonomously correct predictive errors in a changing world, leading to emergent synchrony.