Shield Gemma 2: Developing a Safe Image Classifier for AI

Introducing Shield Gemma 2

• 🚀 Shield Gemma 2 is announced as a new 4 billion parameter image safety classifier, built upon the Gemma 3 model.
• 💡 The model is designed to address the evolving capabilities of open AI models by providing robust safety mechanisms.
• 🔑 Safety is emphasized not as a feature, but as a crucial investment in the AI ecosystem.

How Shield Gemma 2 Works

• 🛠️ The model is built using high-quality labels from a curated mix of synthetic and natural image data across key harm categories.
• 🧠 It leverages in-context learning strategies to develop the 4 billion parameter model on top of Gemma 3.
• ✅ The process involves feeding an input image and a defined, customizable policy into Shield Gemma 2 to receive probabilities indicating policy violations.

Applications and Use Cases

• 🎯 Shield Gemma 2 can function as an input filter for vision-language models (like Gemma 3) or an output filter for image generation systems.
• ⚖️ Key harms addressed include sexually explicit content, violent content, and dangerous content, with customizable policies.
• 📈 Practical applications include online filtering, offline evaluation, model distillation, and adaptation for downstream use cases.

Performance and Customization

• 📊 Evaluation results show Shield Gemma 2 outperforms leading models like Lavagard 7B, GBD4 mini, and out-of-box Gemma 3 4B across key harm types, based on Okmon F1 scores.
• 📝 A prompt instruction template guides the model, starting with problem definitions, incorporating a policy placeholder, and requesting a binary 'yes' or 'no' response for policy violations.
• 💻 The model supports both default and customized policies, demonstrated through Python code examples for loading the model and obtaining predictions.

Community and Future Development

• 🤝 The development of Shield Gemma 2 is attributed to the creativity and resilience of the open-source community.
• ✨ Developers are encouraged to try Shield Gemma 2, adapt it, and provide feedback to help evolve the model further.
• 🌟 The team expresses gratitude and excitement for continued evolution in AI safety.