AI Safety Institutes: Testing Frontier AI Models with Christina Knight

AI Safety Institutes: Global Landscape

• 💡 AI Safety Institutes (AISIs) are government-backed scientific offices, not necessarily regulatory bodies, focused on advancing AI safety science.
• 🌍 There are approximately 10 AISIs globally, with varying mandates; some, like South Korea's, will evaluate models, while others, like the UK's, focus on research.
• 🇬🇧 The first AISI was established in the UK, followed by the US AISI in early November 2023, with a mandate to advance AI safety through guidelines, research, and model testing.

Rationale for Government Involvement

• 🚀 The US AISI was born from the Biden administration's executive order on safe, secure, and trustworthy AI, emphasizing the need for foundational research.
• 💰 Government involvement is crucial because independent researchers often lack the necessary compute resources for robust AI safety explorations.
• ⚖️ Government bodies help balance innovation with responsibility, drawing parallels to historical government roles in developing technologies like those at Bell Labs or in the nuclear field.

AI Safety Testing Methodologies

• 🔍 Red teaming involves probing models for vulnerabilities, either through human experts or automated models, to uncover known and unknown risks.
• ⚠️ Jailbreaking techniques, such as fictionalization or using Unicode, aim to circumvent model safeguards and elicit harmful content or unintended behaviors.
• 📊 Evals (evaluations) assess model capabilities and risks, including safety evals against adversarial attacks and capability evals for tasks like coding or reasoning.
• 📚 Benchmarks are similar to evals but are used publicly to rank models against each other and track advancements in capabilities.

Challenges and Future Trends in AI Testing

• 📉 Evals can become