Rep. Summer Lee Questions AI Surveillance in Schools and Civil Rights

AI Bias and Systemic Inequalities

• 💡 Artificial intelligence algorithms, built and trained by humans, often reinforce existing implicit and explicit biases including classism, sexism, racism, and ableism.
• ⚠️ AI in K-12 schools risks deepening systemic inequalities for marginalized communities, particularly with the rapid expansion of school surveillance AI.

Concerns with AI Surveillance Technology

• 🎯 Facial recognition technology exhibits significant error rates, with a 35% error rate for black girls.
• 🔍 Online chat monitoring and aggression detection tools are layered with racialized assumptions about communication.
• 📚 Much of this surveillance technology extends historical patterns of scientific racism and Eugenics.

Department of Education's Role and Data Infrastructure

• 🏛️ The Department of Education previously provided resources on avoiding discriminatory AI use, highlighting issues like facial recognition misidentifying black students and predictive algorithms flagging them for infractions.
• 📉 Without adequate resources and a functioning Department of Education, schools are ill-equipped to protect student civil rights when deploying AI surveillance.
• 📊 The dismantling of federal data infrastructure hinders the creation of data sets needed to train AI models for equity and fairness.

Mitigating Bias and Ensuring Accountability

• 🛠️ While bias cannot be eliminated, it can be mitigated through guidance, data for training, and industry collaboration.
• 📈 Companies selling AI surveillance to schools face challenges in evaluating their algorithms due to limited data sets, akin to a degraded photocopy.
• 🧑‍🏫 A 2023 survey indicated nearly 40% of teachers reported sensitive student data being shared with law enforcement for use in predictive policing algorithms, with higher rates in schools receiving Title One funding and among special education teachers.
• ⚖️ Without guidance, accountability, and oversight, AI surveillance in schools will exacerbate the school-to-prison pipeline and harm vulnerable communities.