We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 - 28. Join AI and data leaders for insightful talks and exciting networking opportunities. Register today!
University of Michigan researchers recently published a study showing facial recognition technology in schools has limited efficacy and presents a number of serious problems. The research was led by Shobita Parthasarathy, director of the university’s Science, Technology, and Public Policy (STPP) program, and finds the technology isn’t just ill-suited to security purposes, it can actively promote racial discrimination, normalize surveillance, and erode privacy while marginalizing gender nonconforming students.
The study follows the New York legislature’s passage of a moratorium on the use of facial recognition and other forms of biometric identification in schools until 2022. The bill, a response to the Lockport City School District launching a facial recognition system, was among the first in the nation to explicitly regulate or ban use of the technology in schools. That development came after companies including Amazon, IBM, and Microsoft halted or ended the sale of facial recognition products in response to the first wave of Black Lives Matter protests in the U.S.
The University of Michigan study — part of STPP’s Technology Assessment Project — employs an analogical case comparison method to look at previous uses of security technology (CCTV cameras, metal detectors, and biometric technologies) and anticipate the implications of facial recognition. While its conclusions aren’t novel, it takes a strong stance against commercial products it asserts could harm students and educators far more than it helps them.
For instance, the coauthors claim facial recognition would disproportionately target and discriminate against people of color, particularly those in Black and Latinx communities. At the same time, they say facial recognition would promote new codes for dress and appearance and punish students who don’t fit those narrow constraints, causing problems whenever a school relies on the technology to automate activities like attendance taking or lunch sales.
Countless studies have shown that facial recognition is susceptible to bias. A paper last fall by University of Colorado, Boulder researchers demonstrated that AI from Amazon, Clarifai, Microsoft, and others maintained accuracy rates above 95% for cisgender men and women but misidentified trans men as women 38% of the time. Separate benchmarks of major vendors’ systems by the Gender Shades project and the National Institute of Standards and Technology (NIST) suggest facial recognition technology exhibits racial and gender bias and facial recognition programs can be wildly inaccurate, misclassifying people upwards of 96% of the time.
The University of Michigan study’s coauthors also maintain that facial recognition use in schools will create new kinds of student data that will be bought by private corporations. Data collected for one purpose could be used in other ways, making it impossible for students to provide full and informed consent for data collection or control. A legal remedy to this was proposed last week by Senator Jeff Merkley (D-OR) and Senator Bernie Sanders (I-VT) in the National Biometric Information Privacy Act, which would make it illegal for businesses to collect, purchase, or trade biometric information obtained from customers without permission. But most U.S. states currently offer few, if any, protections.
The researchers recommend a nationwide ban on facial recognition in schools and provide policy recommendations for schools that deem the technology “absolutely necessary.” Among other steps, they propose a five-year moratorium on the use of facial recognition technology in schools; convening a national advisory committee to investigate facial recognition and its implications; establishing technology offices to help schools navigate the technical, social, ethical, and racial challenges of facial recognition; and deleting facial recognition data at the end of each academic year or when students graduate or leave the district.
A number of efforts to use facial recognition systems within schools have been met with resistance from parents, students, alumni, community members, and lawmakers alike. At the college level, a media firestorm erupted after a University of Colorado professor was revealed to have secretly photographed thousands of students, employees, and visitors on public sidewalks for a military anti-terrorism project. University of California, San Diego researchers admitted to studying footage of students’ facial expressions to predict engagement levels. And last year, the University of California, Los Angeles proposed using facial recognition software for security surveillance as part of a larger campus security policy.
VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Learn more about membership.