Advancing AI Safety and Trust: Standards, Evaluations, and Global Governance

The AI Reliability Challenge

• 💡 AI currently faces a reliability barrier, separating current capabilities from delivering real-world value, contrasting with past "capability barriers" that were overcome.
• 🎯 Reliability encompasses three key factors: correctness (consistent intended function), product safety (preventing harm), and security (resisting bad actors).
• 🔑 Unlike capability benchmarks that test knowledge, reliability benchmarks assess consistent, safe, and secure operation, which is crucial for building broad trust and unlocking massive societal and business value.
• 📈 Risk management standards are vital for improving reliability by reducing uncertainty, establishing best practices, and enabling effective reasoning about complex, interacting AI systems.

Advancing AI Safety Standards

• 🔬 AI systems require technical standards focused on standardized evaluations and benchmarks, as they are not inspectable like traditional products and need rigorous testing to determine their properties.
• 🚀 Standardized evaluations, such as the MLPerf benchmark, effectively drive progress and foster constructive competition, leading to significant performance improvements and building trust in AI systems.
• 🛠️ Developing industrial-grade benchmarks is complex, requiring well-defined assessment standards, hidden data sets, data refresh mechanisms, and robust governance, contrasting with simpler academic benchmarks.
• 🧩 Addressing the vast scope of AI reliability (across modalities, applications, users, and languages) necessitates shared infrastructure, common core evaluations, and efficient regionalization strategies.

Global South Perspective on AI Governance

• ⚠️ Many AI models are disproportionately trained on English language materials and Western contexts, leading to exacerbated risks and potential harms in Global South regions with diverse languages and cultures.
• 🌱 There is a significant opportunity for an AI Safety Commons to provide shared resources, multilingual safeguards, and contextually relevant evaluations, ensuring that AI safety is a global problem with inclusive solutions.
• 💬 The voice of the Global South is crucial in AI safety decisions, advocating for safety science to be open source and treated as a digital public infrastructure to which all can contribute.
• 🌍 AI systems often break first and most severely in low-resource languages and culturally complex contexts, highlighting the urgent need for a more inclusive and globally representative approach to AI safety.

Operationalizing AI Safety

• ✅ Capacity building for regulators and government officials is essential to proactively understand AI functions, business models, and emerging risks, preventing reactive "knee-jerk reactions" to harms.
• 🤝 Effective operationalization requires multistakeholder input (industry, academia, civil society) and coordination mechanisms like the Network of AI Safety Institutes, ensuring equity in discussing relevant issues for all contexts.
• 📊 Transparency through artifacts like model cards, frontier governance frameworks, and incident reporting is critical across the entire AI value chain to understand and mitigate risks at different levels.
• 📈 The state's procurement policies for AI technologies can play a market-shaping role, setting standards for responsible innovation and operationalizing AI safety guidelines in concrete ways.

Bias, Privacy, and Social Justice

• 🔍 Addressing bias in AI requires better evidence, baseline data, and robust taxonomies of harm specific to diverse contexts, moving beyond anecdotal understandings.
• ⚖️ Privacy by design in AI means embedding protection from the start, considering data curation, accuracy, and safeguards like privacy-enhancing technologies (PETs) while navigating complex trade-offs between utility and individual rights.
• 💡 AI development must acknowledge that privacy is contextual and cultural, and its protection is increasingly linked to social inclusion, as privacy can become a luxury in vulnerable populations.
• 🤝 Technologically-led solutions and government support for democratizing PETs are crucial for fostering trust and enabling innovation without harming individuals, ensuring that AI systems are built for equitable outcomes.