Yann LeCun: Generative AI is the Wrong Path for Physical World Understanding

Current AI Limitations & Misconceptions

• ⚠️ Large Language Models (LLMs) excel in discrete, symbolic language tasks but struggle with the uncertainty and complexity of the physical world, hindering reliable household robots or L5 autonomous driving.
• 🤖 Existing L4 autonomous driving systems are limited to specific areas and times, relying on high-precision maps and massive data, making them unscalable and lacking human-like world models.
• 🚫 Visual Language Models (VLMs) and Visual Language Action (VLA) models are criticized for being script-based and fragile, akin to failed expert systems, as they cannot adapt to novel, untrained scenarios.
• ❌ Generative AI is deemed the "wrong path" for understanding the physical world because it futilely attempts to predict infinite, unpredictable details at the pixel level, which is contrary to how intelligence operates.

The Joint Embedding Predictive Architecture (JEPA)

• 💡 Yann LeCun proposes non-generative Joint Embedding Predictive Architecture (JEPA), which predicts in an abstract representation space rather than raw input, focusing on essential features and ignoring irrelevant details.
• 🧠 JEPA aims to build hierarchical world models, enabling AI to plan at multiple levels of abstraction, from short-term muscle control to long-term strategic goals, much like humans.
• 🎥 AI can learn physical common sense by observing video data through self-supervised learning, similar to how human infants learn about gravity and object permanence.
• ✅ Experiments with V-JEPA 2 trained on video data equivalent to 100 years of footage demonstrated that the model understands basic physical laws, detecting anomalies that violate physics.

LeCun's Cake Analogy for AI Learning

• 🎂 Self-supervised learning is the "cake" (the main body), responsible for AI understanding the world, learning abstract representations, and building world models, requiring no expert behavior observation.
• 🍰 Supervised learning is the "thin layer of icing", allowing AI to quickly master specific skills by imitating human or expert behavior.
• 🍒 Reinforcement learning is the "cherry" (a tiny part), used only for minor fine-tuning due to its extreme inefficiency and high sample cost, making it unsuitable for building complex systems from scratch.

Hardware & Embodied AI Integration

• ⚡ Current AI hardware suffers from high energy consumption due to constant data movement between memory and compute units, unlike the human brain's in-situ processing.
• 🛠️ Future AI hardware needs new technologies (e.g., spintronics, carbon nanotubes) for analog storage and in-situ computation to drastically reduce energy consumption and enable massively parallel, low-frequency systems.
• 🤖 To bridge world models to embodied action, V-JEPA 2.1 uses a two-stage approach: pre-training on natural video for high-level physical understanding, then fine-tuning with minimal data for specific robot kinematics, environment, and interaction models.

Vision for Future AI

• 🚀 The future of AI involves systems that understand the physical world, process any data modality, and build hierarchical world models for planning and reasoning.
• 🎯 This non-generative, planning-based paradigm will move AI beyond "parrot-like" language imitation to "insight-based" world understanding, leading to truly reliable household robots and L5 autonomous driving.
• 💡 LeCun believes this AI revolution is achievable now, with experimental results confirming the viability of learning physical common sense and developing hierarchical planning.