Pick two: The Impossible Triangle of LLM Infra

Understanding Impossible Triangles

• 💡 The impossible triangle is a mental model for understanding developer tooling, where only two out of three competing priorities can be optimized simultaneously.
• 🎯 This concept is seen in various fields, including macroeconomics (impossible trinity), the CAP theorem in databases, and the evolution of frontend frameworks.
• 🔑 The speaker, an economist by background, applies this perspective to analyze developer tooling and frameworks as an investor.

Impossible Triangles in Tech

• 🚀 Examples extend to FinTech (e.g., Mercury, Ramp, Brex competing for features) and data engineering (e.g., DBT and Fivetran merger due to TAM limitations).
• 🧩 Even fundamental cloud components like networking, compute, and storage form an impossible triangle, with various cloud offerings representing combinations of these.
• 📈 These triangles can nest within each other, from broad market segments to specific tech choices like browser automation, web search, and web scraping.

LLM Infrastructure Trade-offs

• 🧠 The impossible trinity for LLM infrastructure comprises Model/API Hosting, Frameworks, and Observability.
• ⚠️ Companies in this space, including model labs and API providers, often face trade-offs and expand into adjacent areas to offer more complete solutions.
• 🛠️ An LLM API platform bundles various jobs to be done; if a platform doesn't offer a feature, developers must build it themselves or wait for the platform to integrate it.

Mastra's Role and Developer Choices

• 🌱 Mastra, initially a TypeScript framework, has predictably expanded its offerings to include observability, evals, and model routing, aligning with the impossible triangle concept.
• ✅ Developers are encouraged to identify the bundle of tools they need by deeply understanding the underlying problems.
• 🧭 The core principle is that problems remain constant, while solutions evolve, providing a robust basis for evaluating new technologies and making informed choices.