Manasi Vartak: Overcoming Enterprise Agentic AI Challenges

Enterprise AI Adoption Challenges

• ⚠️ Many enterprises lead with technology (GenAI) rather than identifying clear business problems, often hindering successful implementation.
• 💡 The generative AI revolution is still in its early stages (three years old), with much experimentation and a low percentage of proofs-of-concept reaching production.
• 🔑 Key challenges include securely connecting LLMs to enterprise data while preserving privacy and managing permissions.
• 🧠 Evaluating non-deterministic AI systems is a significant problem, requiring new approaches to build trust and ensure reliable decision-making.
• 🚀 Change management is crucial, as employees need training to effectively prompt and integrate AI into their daily workflows.

Private AI Solutions

• ✅ Private AI involves using an organization's own models, data, and infrastructure (data centers or cloud VPCs) to keep sensitive information within secure boundaries.
• 🔒 This approach is vital for industries with highly sensitive data that cannot leave an organization's controlled environment, such as financial services.
• 📊 The decision to use on-premise infrastructure versus cloud services often depends on scale and regulatory requirements, with on-prem becoming cost-effective for large-scale inference (e.g., over 100 models).
• 🌍 Geographical regulations (e.g., in APAC or the Middle East) often mandate data residency, requiring AI models to operate locally to comply with laws and reduce latency.

Data and AI Governance

• 📌 Data governance is foundational for AI governance, addressing data lineage, source, and the masking of Personally Identifiable Information (PII).
• ⚖️ AI governance extends beyond data, incorporating ethical considerations, bias detection, and integration with existing risk management processes.
• 🔍 Organizations must carefully consider what data is sent to third-party LLMs and ensure robust governance frameworks are in place for sensitive information.

Evolving AI Workflows and Skills

• 🛠️ Building with AI requires different workflows, especially for non-deterministic outputs, necessitating new methods for writing test cases and evaluations.
• 📈 Teams should build for future model capabilities (e.g., 80-90% accuracy) rather than current limitations, as the underlying technology rapidly improves.
• 🧑‍💻 Essential skills for AI builders include rapid prototyping and strong user experience (UX) design, as natural language interfaces are still evolving.
• 🎯 The AI landscape is creating a bifurcation of skills: deeply technical roles for system internals and business experts leveraging AI for quick workflows.
• 🌐 Data teams are shifting from direct analytical problem-solving to enabling self-service for business users through embedded AI capabilities.

The Future of AI and Human Interaction

• 🤖 AI agents are poised for significant development, moving beyond simple assistants to more autonomous and reasoning-capable systems.
• 🤝 Human oversight remains critical for AI systems, especially where the cost of error is high, requiring humans in the loop for review and auditing.
• ✨ Future innovations include specialized models for specific domains (e.g., healthcare, drug discovery) and improving the human-AI interface to enhance productivity without increasing "AI slop."
• 🗣️ The authority of human creators and the development of "expert marketplaces" for information may become more important in an AI-saturated content landscape.