Prof. Melanie Mitchell: Investigating Abstract Reasoning in Humans and Machines

Evaluating AI Cognitive Capacities

• 💡 The speaker highlights challenges in assessing AI's cognitive abilities, noting that Turing tests can lead to the Eliza effect, where human attributes are projected onto AI systems.
• 🎯 Current AI benchmarks are often saturated, but suffer from issues like data contamination, models taking shortcuts, and a lack of construct validity.
• 🧠 Explanations from "reasoning models" are frequently unfaithful to their actual internal operations, making it difficult to trust their purported reasoning processes.

Cognitive Science Evaluation Principles

• 🔬 A more objective evaluation approach involves adapting experimental methodologies from cognitive science to study AI systems.
• ✅ Key principles include being aware of anthropomorphic cognitive biases, designing control experiments to identify alternative strategies, and testing robustness and generalization with novel stimuli variations.
• 📊 It's crucial to distinguish between a system's performance and competence and to analyze failure types to understand the underlying mechanisms, rather than just reporting accuracy scores.

Analogical Reasoning Robustness

• 🔑 Initial studies suggested GPT-3 and GPT-4 outperformed humans in analogical reasoning tasks.
• ⚠️ However, further robustness testing with variations (e.g., changing alphabets, answer positions, paraphrasing stories) revealed that GPT models were significantly less robust than humans.
• 🧩 AI systems often exploited superficial features like syntactic similarity and answer ordering, which humans did not rely on for their reasoning.

Conceptual Abstraction in ARC

• 🚀 The Abstraction and Reasoning Corpus (ARC) was designed to measure human-like core knowledge priors and general fluid intelligence in AI.
• 📈 While AI models like 03 achieved high accuracy on ARC, analysis of their stated rules showed they frequently used unintended numerical comparisons or spurious associations.
• 🔍 High accuracy can overestimate a model's true abstract reasoning capabilities, whereas low accuracy might underestimate its competence due to performance constraints.

Importance of Human-AI Alignment

• ✨ Understanding the alignment between human and AI understanding is critical for assessing trustworthiness, safety, and interpretability in real-world applications.
• 🚫 Accuracy alone can mask the exploitation of superficial features or unhuman-like reasoning, leading to systems that don't generalize as expected.
• 👏 The AI community should prioritize replication and incremental extensions of prior work, focusing on better evaluation of existing benchmarks to truly understand how systems function, rather than solely pursuing harder benchmarks.