Jack Morris on AI Breakthroughs, Research, and Open Source vs. Closed Source

The State of AI Research

• 💡 AI research is characterized by a split between public academic work and private, often unpublished, industry research.
• 🧠 While AI models are becoming more capable, humans are still essential for driving improvements and innovation.
• 🚀 The pace of AI progress is unpredictable, with breakthroughs often occurring in unexpected areas rather than anticipated ones like AI agents.

Evaluating AI Model Progress

• 📊 Model evaluation primarily relies on testing against specific datasets (e.g., SWEBench for coding, math Olympiads) and human ranking systems like ELO scores.
• ⚠️ Benchmarks can be misleading, with some open-source models showing high scores but performing less effectively in real-world applications.
• 🧩 AI models are continuously being patched to address specific failures, such as riddle-like questions, which can make them appear more intelligent without necessarily reflecting deeper understanding.

Training and Data in AI

• 🛠️ AI models are trained through supervised learning (copying data) and reinforcement learning (rewarding desired outcomes), with reinforcement learning showing significant promise for improving model capabilities.
• 📚 The availability and quality of data are crucial differentiators, with companies like Anthropic collecting vast amounts of unique data (e.g., old books) to gain an edge.
• 🌐 Open-source models, like those from Chinese labs, offer transparency and accessibility, but proprietary data and infrastructure give US-based labs a potential advantage.

The Future of AI Development

• 🎯 Future AI advancements may stem from personalization and online learning, where models are tailored to individual users or companies and continuously update based on interactions.
• 📈 The drive for cutting-edge AI is fueled by a competitive race among labs, leading to higher stakes and a focus on significant, often commercially valuable, innovations.
• 🤝 While financial incentives are substantial, many researchers are motivated by the pursuit of scientific frontiers and the desire to be part of significant technological advancements.

Open Source vs. Closed Source AI

• 🔒 Closed-source models allow companies to protect proprietary training methods and data, but can lead to user frustration when models are updated and personalized training is lost.
• 🌐 Open-source models foster collaboration and accessibility, enabling researchers to build upon existing work, but may reveal sensitive training details.
• 🔍 AI researchers often analyze and build upon existing models, with open-source contributions playing a vital role in scientific progress and understanding.