How Will Mech Interp Help Make AGI Safe?

The North Star of Interpretability

• 💡 The primary goal of interpretability is to ensure AGI does not harm humanity, rather than purely scientific understanding.
• 🎯 It aims to make aligned AGI, which means preventing models from acting disastrously.

How Interpretability Aids AGI Alignment

• 🔍 Interpretability can help rigorously identify dangerously misaligned models, as current evaluation methods are often insufficient.
• 🛠️ It can enable highly capable AIs to automate the R&D process for safer AIs, by providing checks and balances.
• 🌱 Interpretability contributes to basic scientific understanding of neural networks, including their psychology and generalization, which is crucial for safety.

Defining and Achieving Alignment

• 🔑 The focus is on intent alignment (model doing what developers intended), which is considered more tractable than value alignment.
• 🧠 Understanding model psychology is viewed from a utilitarian perspective, aiming to improve user experience and address public pressure.
• ✅ Interpretability can provide clearer and richer feedback on system alignment, helping to debug issues and improve training processes.

Research Focus and Feedback Loops

• 🚀 Prioritize research that is hard to fake, such as real-world user experience improvements or contrived demos that reveal deep, hard-to-fix issues like blackmailing.
• 🔬 Studying naturally occurring misalignment (e.g., model fix tests) is highly valuable for gaining insights into how misalignment manifests.
• 💡 Basic science questions should target tractable and interesting mysteries that offer predictive power for real-world problems.

Confidence and Precision in Alignment

• 📈 Achieving high confidence (e.g., 90-99%) in detecting misalignment is a key goal for AI control.
• 📊 Proving misalignment (e.g., a model is driven by self-preservation) does not necessarily require high precision or completeness.
• 🧩 Precision refers to perfectly capturing a concept, while completeness means not missing any factors or concepts.