AI Is NOT What You Think: We Finally Cracked the "Black Box" of Deep Learning!

Understanding AI's "Black Box"

• 🧠 Modern AI models are often referred to as "black boxes" because their internal decision-making processes are not fully understood, even by their creators.
• 🔍 Researchers are actively working to peek inside these complex systems to gain insight into how they truly operate and "think."

The Phenomenon of Grokking

• 💡 In 2021, OpenAI researchers accidentally discovered "grokking" while training an AI model to perform modular arithmetic, similar to a clock face.
• 🚀 Grokking describes a sudden and spontaneous shift where an AI model transitions from mere memorization of answers to achieving perfect generalization after thousands of training steps.
• 🎯 Initially, the model only memorized, but after being left to run, it unexpectedly learned to generalize and solve new problems.

Unveiling Internal Mechanisms

• 🔬 Through mechanistic interpretability, researchers like Neil Nanda investigated the grokking model's internal activity to understand its sudden leap in performance.
• 🌊 They observed organized rhythmic patterns resembling sine waves and perfect circular loops in the activity of individual neurons, which were absent before grokking.
• ✨ The AI spontaneously rediscovered trigonometry, specifically the "sum of angles" identity, to represent numbers as waves and transform addition problems into geometry problems.

Mechanistic Interpretability in Action

• 🛠️ The insights gained from understanding grokking led to the development of new "detective tools" for probing the minds of larger AI models.
• 💬 Anthropic researchers utilized these tools on Claude Haiku, discovering it uses complex six-dimensional geometry to handle tasks like line breaks.
• 👽 This research suggests that AI's internal thought processes involve alien mathematical structures and high-dimensional geometry, fundamentally different from human cognition.