AI Bias and Legal Dilemmas: A Discussion with Ryan Calo

Historical Context of Bias in Law

• ⚖️ Historically, US anti-discrimination laws primarily focused on intentional discrimination, which is a poor fit for AI systems.
• ⚠️ A doctrine of disparate impact exists, allowing claims if a system has a discriminatory effect on a protected class, even without intent, but this is narrowly applied.
• 🔒 Protected classes typically refer to demographic groups like gender and race, on which selection is not allowed.

Bias in Computer Systems and Early AI

• 💡 Early work on bias in computer systems, dating back to the mid-90s, highlighted how designer biases can be reflected in systems.
• 🚗 Examples include Uber's algorithms recreating historical biases in service to certain neighborhoods, despite being touted as a solution.
• 🔍 Research has shown search engines and other systems exhibiting biases against women and Black women, a pattern seen later in resume screening and facial recognition.
• 🧠 Bias can be inevitable if viewed as the preconceptions of system designers, but becomes deeply troubling when systems operate to discriminate.

AI's Amplification of Bias and Legal Challenges

• 🤖 AI systems can amplify existing societal biases, leading to discriminatory outcomes in areas like hiring, admissions, and criminal justice.
• ⚖️ In criminal justice, facial recognition has shown poorer performance for Black individuals, and algorithms for pre-trial detention have been found to disproportionately label Black individuals as more dangerous.
• ❓ A key tension exists between improving upon a flawed system with AI and the ethical imperative to eradicate bias altogether, questioning