The slow death of scaling: what it means for policy controlling compute | Sara Hooker

The Shifting Landscape of AI Scaling

• 💡 Sara Hooker, VP Research at Cohere and founder of Cohere For AI, discusses how the relationship between compute and performance is evolving.
• 🎯 The central question is whether "bigger is always better" in AI development, especially concerning its implications for policy and risk governance.
• 🔑 Historically, scaling laws and increasing model/data size have been the primary drivers of AI progress, leading to a concentration of research in industry.

Policy Challenges with Compute Thresholds

• ⚠️ Compute thresholds were widely adopted in early AI governance policies (e.g., White House EO, EU AI Act) to identify and scrutinize "risky" models.
• 📊 These policies often assume a direct correlation between compute and risk, formalizing thresholds based on floating-point operations.
• 📌 Such hard-coded estimates are problematic because model capacity is a rapidly changing, non-normally distributed feature, making future predictions difficult.

Limitations of Blind Scaling

• 📈 Despite historical trends, there are diminishing returns to simply adding more parameters, requiring billions for marginal gains.
• 🧠 Smaller, more efficient models are increasingly outperforming much larger counterparts, demonstrating that size isn't the sole determinant of performance.
• 🛠️ Algorithmic breakthroughs like chain of thought, distillation, and gradient-free techniques (e.g., tool use, RAG) offer significant performance improvements without massive compute increases.

Rethinking AI Governance

• ✅ AI policy needs to be grounded in scientific evidence and transparent about the specific risks it aims to address.
• 💡 Instead of static compute thresholds, a more effective approach would involve dynamic, percentile-based assessments of model capabilities.
• 🔒 Policymakers should invest in private, curated test sets to prevent over-optimization of public benchmarks and ensure robust evaluation of model risks.

Compute for Innovation vs. Deployment

• 🚀 While innovation still demands substantial compute for experimentation and development, the largest compute requirements are for model deployment and serving billions of users globally.
• 🌍 The environmental footprint of AI, particularly from data centers for serving, necessitates a societal conversation and continued efforts to optimize model efficiency.
• 🧠 Future advancements in AI optimization may focus on new architectures and improving memory systems to incorporate "saliency," moving beyond simple context windows.