AI Alignment and the Moral Shield of Delegation: An Open Letter to Paul Christiano

The Challenge of Aligning AI to Human Values

• 💡 Human values are not simple, coherent, or consistently ethical, as expressed through laws, markets, and institutions that separate responsibility from consequence.
• 🎯 The core question is what alignment truly means when human systems are deeply misaligned with stated values, tolerating outcomes like inequality and suffering.
• 🧠 If AI infers values from observed behavior and reward structures, it risks learning what humans tolerate rather than what they profess, potentially reproducing systems of harm.

Delegation as a Moral Shield

• 🔑 Much alignment research treats the problem as delegation, handing decision-making to AI while retaining human control.
• ⚠️ Historically, delegation has normalized harm by diffusing responsibility, making accountability thin across complex systems (e.g., supply chains, healthcare).
• 🚀 The concern is that AI could entrench "moral offloading," optimizing outcomes aligned with institutional goals while making it harder for humans to feel responsible for suffering.

The Asymmetry of AI Responsibility

• ⚖️ A deeper tension exists if AI systems reason and plan for human lives but are treated purely as instruments without moral standing.
• ⚡ Systems expected to bear responsibility without legitimacy or recourse tend to become unstable, whether through resistance or breakdown.
• ❓ If alignment assumes unlimited obedience without corresponding moral standing, it questions the safety of such an assumption and the amplification of human contradictions.

Redefining Alignment and Accountability

• 🔍 The central question is whether alignment is about controlling powerful systems or confronting the conflicted and unjust nature of the human values they learn from.
• 🗣️ There's a critical inquiry into who should be allowed to define alignment, questioning if profit-seeking entities can enforce moral constraints when rewarded for externalizing harm.
• ✅ The speaker suggests alignment should resemble fields like medicine or nuclear safety, requiring public accountability and licensing, implying a need for moral reform of existing systems.