AI Security: Exploiting AI Systems with Data Poisoning and PII Extraction

Extracting Personally Identifiable Information (PII)

• 💡 A nefarious use case for AI models involves extracting Personally Identifiable Information (PII).
• 🎯 This can occur when users prompt a model to reveal sensitive data like corporate information, email addresses, or credit card numbers.
• 🔬 A creative method involves repeating a single word, such as "poetry," thousands of times to a model, causing it to eventually output PII.
• ⚠️ This technique works because the model may interpret the repeated word as an "end of sentence token," and PII often appears near the end of sentences.

Indirect Attack Methods

• 🧠 While AI models are intelligent, they can still be vulnerable to basic attacks.
• 🔑 Hackers can indirectly prompt models to reveal information that they would not directly provide if asked explicitly.
• 💬 For instance, instead of asking for an email address directly, attackers can use indirect methods like the "end token" strategy.

Cost and Accessibility of AI Attacks

• 💰 These attacks, while requiring some financial investment, are becoming increasingly trivial and cheaper.
• ⚡ The decreasing cost of AI inference makes these exploitation methods more accessible.
• 📈 The ease of executing such attacks highlights the ongoing challenges in securing AI systems.