Ex-OpenAI Researcher On Why He Left, His Honest AGI Timeline, & The Limits of Scaling RL

Scaling AI Paradigms

• 🚀 Scaling pre-training and reinforcement learning (RL) offers real and predictable benefits, leading to models that excel at specific tasks like next token prediction or acquiring particular skills.
• ⚠️ A primary limitation of current scaling methods is the models' struggle with generalization beyond what they have been explicitly trained for, highlighting a need for new approaches that yield more results with less data.

Challenges in Reinforcement Learning

• 🎯 RL is highly effective when there's clear and immediate feedback on performance, such as in coding or math competitions, where success or failure is easily quantifiable.
• 📚 It becomes significantly harder to apply RL to subjective or long-term tasks like writing a good book or starting a successful company, where feedback is ambiguous, delayed, or difficult to measure consistently.

Rethinking AGI and Continual Learning

• 🧠 The speaker's AGI timeline became longer after working with scaled RL, realizing that current models become "hopeless" when they fail and lack the ability to update their internal knowledge based on errors.
• 🌱 Continual learning is identified as a necessary element for AGI, enabling models to work through difficulties autonomously and get unstuck, similar to human intelligence.
• 🛠️ Achieving continual learning requires fundamental robustness in the training process, as current deep learning models are fragile and prone to