LlamaCon 2025 Opening Session: Llama 4 API & Open-Source AI Innovation

The Vision for Open-Source AI

• 💡 Two years ago, open-source AI was considered a "dream" with concerns about financial viability, distraction, safety, and performance.
• ✅ Meta built on open-source foundations from its early days, with engineers contributing to web, mobile, and AI stacks.
• 🚀 Open source is now recognized as an essential part of AI development, offering safety through auditing, frontier performance, and customization for specific use cases.

Llama 4 Innovations & API

• 🧠 Llama 4 offers more performance in a smaller binary, is the first open-source multimodal model (trained on photos and text), and supports 200 languages natively.
• 💻 It features a huge context window capable of holding entire codebases or complex documents, and includes models like Scout (single H100) and Maverick (17B parameters, used in Meta AI).
• 🛠️ The new Llama API aims to be the fastest and easiest way to build with Llama, offering speed, ease of use, customization, and no lock-in, with support for Python, TypeScript, and OpenAI SDK.
• ⚙️ Key features of the API include a chat completion playground, image input, JSON schema for structured responses, and tool calling (preview feature).

Diverse Applications & Meta AI

• 🌍 Llama is deployed in various sectors: Booze Allen and ISS for documentation access, Sophia and Mayo Clinic for healthcare paperwork and diagnosis, and Farmer Chat for agricultural information in Africa.
• 🛒 Other applications include Kavak for customer support in used car marketplaces and AT&T for analyzing daily customer service transcripts to identify bugs.
• 📱 The new Meta AI app focuses on a natural voice experience with low latency, high expressiveness, personalization (connecting Facebook/Instagram), and experimental full duplex voice.
• 🕶️ Meta AI also integrates with Ray-Ban glasses, allowing multimodal interactions and questions about the user's surroundings via a voice interface.

Advancing Model Efficiency & Customization

• ⚡ Inference efficiency was a key design goal for Llama 4, utilizing a mixture of experts architecture and specific sizing for models like Maverick to fit on a single host with FP8 weight quantization.
• 🚀 New runtime features include speculative decoding (1.5-2.5x faster token generation) and page KV for handling long queries efficiently.
• 🤝 Partnerships with Cerebras and Groq aim to deliver even faster inference speeds through the Llama API, with early experimental access available.
• 🔄 The Llama API enables fine-tuning for product use cases, giving users full agency over custom models and the ability to download and run them anywhere, preventing vendor lock-in.

Future Trends & Developer Advice

• 🔬 Meta's FAIR team is focused on visual AI tools like Locate 3D (labeling 3D objects with text) and SAM 2 (object detection for image/video), with SAM 3 coming soon, offering text-based object labeling.
• 🗣️ Voice interaction is predicted to become a dominant paradigm for AI, moving beyond text-heavy use cases, especially with wearable devices like smart glasses.
• 💡 Developers are advised that it's "day zero" for AI, with many applications yet to be invented, emphasizing the importance of data advantage, iterative improvement, and building custom models.
• 🤝 The open-source ecosystem fosters rapid progress, community building, and the ability to mix and match different model capabilities through techniques like distillation, leading to better and cheaper solutions.