Tom Griffiths: Logic, Probability, and AI in The Laws of Thought

Foundations of Thought

• 💡 The concept of "laws of thought" refers to general principles governing intelligence, spanning abstract principles and concrete brain mechanisms.
• 🧠 Early philosophers like Aristotle and Leibniz sought to formalize thought, with Leibniz envisioning arithmetic as the basis for machine intelligence (precursor to vector embeddings).
• 🔑 George Boole advanced this with his algebra, providing a mathematical framework for logic, initially focusing on true/false statements.

Logic and Probability

• 🎯 The shift from certain logic to probabilistic reasoning was a major development, pioneered by Boole, Bayes, and Laplace.
• 📈 This allowed for reasoning with degrees of belief and updating them based on new information, extending classical logic where conclusions are certain.
• 📊 Bayesian probability offers an ideal solution for inductive problems, describing how minds should update beliefs, even if humans don't always achieve this perfectly.

Human vs. AI Cognition

• 🤖 Large Language Models (LLMs) often struggle with tasks like arithmetic or generating random numbers, similar to human limitations, despite being computers.
• 💡 A key difference is inductive bias: humans learn language from vastly less data (e.g., 5 years) compared to the immense datasets required for LLMs (e.g., 5,000-50,000 years).
• 🧠 Current AI systems exhibit "jagged intelligence," excelling in one area but failing spectacularly in closely related ones, unlike the generalizability of human cognition.

Resource Rationality

• ⚡ Human "irrationality" can be understood through resource rationality, where cognitive shortcuts (heuristics) are adaptive responses to finite time, energy, and information.
• ✅ These heuristics and biases are not necessarily flaws but optimal strategies for solving problems given limited resources.
• 🚀 Strategies like sampling possible outcomes or setting sub-goals help humans make progress on complex problems without infinite computation.

Neural Networks and Learning

• 🔬 The idea of representing concepts as points in space (rather than logical rules) emerged from work on categories, leading to the development of neural networks.
• 🛠️ Neural networks compute by transforming vectors of values, mapping points from one space to another, with multi-layered networks handling more complex computations.
• 🌱 Meta-learning approaches aim to give neural networks "head starts" by manipulating initial weights, allowing them to learn from less data and develop more human-like inductive biases.

Marr's Levels of Analysis

• 🧩 David Marr's three levels (computational, algorithmic, implementation) explain why there isn't a single, unified theory of mind.
• 🤝 Theories at each level (e.g., logic/probability at computational, neural networks at algorithmic, brain cells at implementation) can be individually correct and mutually compatible.
• 🔍 A complete understanding requires relating these different levels, recognizing that there can be multiple algorithmic solutions for a single computational problem.