Agentic AI Security Risks: Threat Modeling with Shaked Reiner

Understanding Agentic AI

• 💡 Agentic AI refers to systems where an LLM controls the program's flow, enabling autonomous actions like web browsing or code execution.
• ⚡ Unlike traditional LLMs, agentic systems can perform actions in the real world, making vulnerabilities more severe.

Threat Landscape of Agentic AI

• 🎯 Research systematically maps the threat landscape, focusing on how known LLM vulnerabilities apply to agentic systems.
• ⚠️ Traditional access vectors (server-level attacks) remain relevant, alongside a new surface of LLM-based attacks like prompt injections and model manipulations.

Identity and Access Management Challenges

• ❓ The industry is still determining how to classify AI agents (users, machines, bots), complicating traditional IAM.
• 🔑 Granting agents permissions, access tokens, and accounts is necessary for them to act, but requires careful consideration of their identity and access levels.
• 📈 Overprivileging agents is a severe risk, as an attacker with limited access might manipulate an agent with broader permissions to perform actions they couldn't otherwise.

Lifecycle Management and Best Practices

• 🔒 Organizations must trust the base LLM models and the developers of agents, or carefully monitor their own custom-developed agents.
• ⚙️ Agent behavior can be drastically altered by changing instructions or system prompts, necessitating monitoring of these configuration files.
• ✅ Key recommendations include: never trust the LLM, validate and sanitize LLM outputs, limit the LLM's scope of action, apply the principle of least privilege, manage credentials diligently, and implement robust security monitoring and threat detection.

Future of Agentic AI and Cybersecurity

• 🚀 The pace of AI development is extremely fast, making it a moving target for security professionals.
• 🔮 Agentic AI will likely look vastly different in a year or two, with enhanced functionality and productivity.
• ⚔️ New, advanced attack vectors exploiting AI agents are expected to emerge, requiring novel security measures.