Q&A: Dr. Joy Buolamwini on AI, Surveillance, and Consent

Dr. Buolamwini's Journey to AI Activism

• 💡 Dr. Buolamwini initially wanted to make cool tech and believed technology was apolitical, viewing it as an escape from social sciences.
• 🎯 A rude awakening occurred when her Aspire mirror art project couldn't "see" her, confronting her with the inherent biases in the field she loved.
• 🔑 Despite warnings about the risks of researching discrimination and bias, she couldn't ignore the "coding in a white mask" experience, leading her to shift from maker to critic.
• 👏 Support from figures like Cathy O'Neil (author of "Weapons of Math Destruction") and Timnit Gebru (who faced repercussions for her work on LLM limitations) provided courage and validation.

Addressing Bias in AI Systems

• 🔬 Historical data, such as medical records, often contains significant biases (e.g., women excluded from clinical trials until 1993), leading to biased AI outcomes.
• ⚠️ A study using ChatGPT for mental health prediction found Black and Hispanic individuals were nine times more likely to be predicted to commit suicide, despite the predictions being inaccurate.
• 🧩 The Algorithmic Justice League emphasizes acknowledging limitations and ensuring AI tools are "fit for purpose" by understanding data composition, rather than treating AI as "mystery meat."

AI's Impact on Employment and Public Safety

• 📈 AI is influencing hiring processes, sometimes leading to hiring freezes or the replacement of human workers, as seen with NETA's chatbot failure which gave harmful advice for eating disorders.
• 🚨 The deployment of police body camera footage with AI for correlating active warrants raises concerns about a "backdoor to the Fourth Amendment" and the need for public deliberation.
• ✅ Awareness campaigns and research led to Amazon, Microsoft, and IBM stopping sales of facial recognition to law enforcement, highlighting the power of resistance against unchecked deployment.

Automation and Ethical Considerations

• ⚽ In sports, AI automation aims to reduce biased human officiating, with examples like tennis using automated line calls with limited human challenges.
• ⚖️ The discussion around semi-automation versus full automation in sports emphasizes the need for a balance of technological assistance and human decision-making.
• 🌐 A core question for all AI deployment is: who decides if it is "fit for purpose" and for whom before it is scaled, especially given the diverse impact on careers and industries.