RAG Agents in Prod: 10 Lessons We Learned — Douwe Kiela, creator of RAG

The Enterprise AI Paradox

• 💡 Despite a $4.4 trillion opportunity, only one in four businesses derive value from AI, creating a paradox where seemingly easy tasks for humans (like context) are hard for AI.
• 🧠 Large Language Models (LLMs) excel at complex tasks like code generation and mathematical problems, but struggle with contextual understanding, which is crucial for real-world enterprise applications.

Building Robust RAG Systems

• 🛠️ LLMs represent only 20% of an enterprise AI deployment; the surrounding RAG pipeline and system are more critical for solving problems effectively.
• 🎯 Enterprises should prioritize specialization over general AGI to effectively leverage their unique internal expertise and institutional knowledge.
• 📊 A company's data is its long-term moat, and the ability to make AI work on noisy data at scale is key to unlocking differentiated value.

Scaling to Production

• 🚀 Design for production from day one, as pilots are deceptively easy, but scaling to thousands of users, documents, and diverse use cases with security and compliance is far more complex.
• ⚡ Prioritize speed over perfection by iterating quickly and gathering feedback from real users early, rather than striving for a perfect initial launch.
• ✅ Ensure engineers focus on delivering business value by abstracting away repetitive tasks like chunking strategies, allowing them to concentrate on differentiated solutions.

Driving Adoption & Impact

• 🤝 Make AI solutions easy to consume and deeply integrate them into existing workflows to maximize actual usage and user stickiness.
• ✨ Aim to "wow" users quickly by demonstrating immediate value, such as uncovering long-lost critical information, to foster adoption and advocacy.
• 🔍 Move beyond just accuracy to address inaccuracy through observability, audit trails, and strong attribution in RAG systems, especially in regulated industries.

Ambition in AI Deployment

• 📈 Be ambitious in AI projects, targeting significant ROI and business transformation rather than settling for low-impact "gimmicks" like basic HR Q&A.
• 🌍 Recognize that AI is at a transformative moment, offering a unique opportunity for enterprises to drive profound societal and business change.