AI Auditing and Evaluation: Beyond Benchmarking with Inioluwa Deborah Raji

The Evolving Role of AI Evaluation

• 💡 AI systems are increasingly deployed in critical domains like criminal justice and healthcare, leading to real human costs when they fail.
• 📌 Historically, consumer protection movements have driven product safety evaluations, a model relevant to today's AI landscape.
• 🚀 AI evaluation has shifted from merely ranking algorithms to playing a broader role in product deployment, documentation, and legal evidence.

Auditing for Accountability

• 🎯 The AI auditing process involves identifying harms, evaluating against standards, communicating results, advocating for change, and ensuring legal accountability through consequences.
• 🔬 The Gender Shades project demonstrated how evaluating facial recognition bias led to significant changes in documentation, procurement, and legal actions.
• 📈 Postmarket surveillance methods, inspired by vaccine adverse event reporting, can statistically identify disproportionately harmed subgroups in AI systems.

Beyond Traditional Benchmarking

• ⚠️ Traditional AI benchmarks often suffer from a construct validity challenge, failing to accurately represent real-world performance or generalizability.
• 🧠 Realistic evaluations, like using patient notes for clinical LLMs, reveal that models perform differently and often worse than in idealized test scenarios.
• 🧩 AI deployments should be viewed as policy interventions, where experiment design choices significantly impact user responsiveness and the accurate measurement of causal effects.

Operationalizing AI Audits

• 🛠️ Policy engagement is crucial for translating new evaluation methods into effective AI audit implementation, addressing data access and standard setting.
• ✅ Model cards and other documentation practices are becoming essential for clinical decision support tools and government AI use, ensuring transparency and accountability.
• 🌐 Multi-disciplinary collaboration and AI Safety Institutes are vital for developing the necessary technical and institutional infrastructure for safe and widespread AI adoption.