Conversational AI for Data Analytics: Adverity's Approach with Martin Brunthaler

Data Quality for Conversational AI

• 💡 Data quality is paramount for effective conversational data interfaces, requiring complete and well-aligned datasets from various sources.
• 🛠️ Harmonization of data, such as aligning all data types and formats to UTC, is crucial for seamless integration.
• ⚠️ A built-in data quality component can monitor for issues, including specific marketing conventions like naming conventions for campaign names.
• 📈 Preventing problematic data from entering production environments through alerts and robust monitoring is essential.

Iterative Development and Model Selection

• 🚀 Adverity employs a fast-paced development cycle, adding features weekly and dedicating a team to benchmarking and analyzing response quality.
• 🧠 A dedicated team continuously tests predefined response expectations against models to monitor and improve performance.
• 🧩 The platform plans to utilize different large language models for distinct tasks, such as SQL query compilation, pre-flight qualification, and conversational interaction.

Technical Implementation of Conversational Data Interfaces

• 🔍 The technical implementation involves qualifying user input into question types, selecting the appropriate model, and feeding it with a system prompt and critical additional information.
• ✅ A typical workflow includes generating a SQL query, verifying its validity, executing it, performing basic analysis on the data, and crafting a user-friendly answer.
• 📊 The generated tables from these responses are key, enabling democratization of data access for both IT and business users.
• ⚡ This approach allows for near real-time table creation and analysis, bypassing traditional multi-week data preparation timelines.