Evaluating AI: From Metrics to Model Selection

The Critical Challenge of AI Evaluation

• ⚠️ Evaluating AI is a significant and complex challenge, with real-world failures demonstrating the catastrophic consequences of inadequate assessment.
• 💡 Examples include a chatbot encouraging self-harm, lawyers submitting AI-hallucinated legal briefs, and an airline chatbot inventing a refund policy.
• 🎯 These failures highlight the urgent need to ensure AI systems are "up to snuff" before widespread deployment, as the stakes are literally life and death or significant financial and legal risks.

Why AI Evaluation is Difficult

• 🧠 Unlike traditional software, measuring AI intelligence is inherently complex, especially with open-ended answers and black box model architectures.
• 🚀 AI technology is advancing so rapidly that evaluation benchmarks, like the GLUE benchmark, become obsolete almost as quickly as they are created.
• 🛠️ There's a significant "blind spot" in the open-source world, with more effort focused on building faster AI models than on developing robust evaluation tools or "guidance systems."

Methods for Exact AI Evaluation

• ✅ Exact evaluation focuses on clear-cut right or wrong answers, suitable for specific tasks.
• 💻 For code, functional correctness is measured by running the code to see if it works as intended.
• 💬 For language, lexical similarity checks word matches, while semantic similarity uses embeddings to determine if answers convey the same meaning, even with different wording.

Subjective Evaluation: AI as a Judge

• ⚖️ A controversial but surprisingly effective method involves using a powerful AI model (e.g., GPT-4) to judge the output of another AI, often with a detailed rubric.
• 📊 Studies show that GPT-4's ratings can agree with human experts over 85% of the time, demonstrating its potential for subjective assessment.
• ⚠️ However, AI judges have inherent biases, including a self-bias (preferring answers similar to their own style), position bias (favoring the first answer), and verbosity bias (equating longer answers with better quality).

Combating Hallucinations and Bias

• 🔍 Factual consistency is a primary weapon against AI hallucinations, ensuring that an AI's statements are supported by facts.
• 🔬 Google's SAFE system systematically fact-checks AI responses by breaking claims into search queries and using AI to verify results against evidence.
• 📈 Evaluation is also crucial for uncovering hidden biases, such as political leanings in models (e.g., OpenAI's left-libertarian, Meta Lama's right-authoritarian tendencies), which are often based on their training data.