AI hiring tool disproportionately screened out Asian applicants, study finds

A hiring algorithm promoted as a way to make hiring more objective disproportionately screened out Asian job applicants, according to a Stanford-led study. Candidates of Asian descent faced the largest cumulative shortfall in recommendations among the racial groups analyzed.

The study examined assessments administered through pymetrics, a hiring platform now owned by Harver. Roughly 29,000 more applications from Asian job seekers would have advanced if they had been selected at the same rate as the most-selected racial group for each position.

Earlier audits did not detect significant racial disparities because they evaluated hiring outcomes in aggregate rather than by individual job. A racial group could be favored in some positions and disadvantaged in others, allowing harms at specific jobs to disappear when results were combined into a single analysis.

A position-by-position review found adverse impact affecting Asian and Black applicants across multiple jobs. The study used the four-fifths rule, a common employment discrimination benchmark that flags potential adverse impact when one group’s selection rate falls below 80% of the rate for the highest-selected group.

The findings challenged assumptions that removing resumes and demographic information automatically reduces bias. Pymetrics evaluates applicants through behavioral games designed to measure traits such as attention, risk tolerance and decision-making. The characteristics measured by those assessments could still correlate with race, producing unequal outcomes even when demographic information was not explicitly used.

Co-author Kathleen Creel said the findings show why hiring systems should be evaluated at a more granular level. The paper argues that employers and regulators risk overlooking discriminatory outcomes when they rely on broad audits rather than examining how algorithms perform in individual jobs.

The study also identified a pattern the authors called “systemic rejection,” in which applicants screened out by one employer were more likely to be rejected by others using the same technology.

Among applicants who submitted four applications through employers using the platform, 10% were rejected from every position. The rate exceeded what would be expected if each employer had made independent decisions. The study also found that 4% of applicants who hypothetically applied to 10 positions would have been rejected by every position, illustrating how the same screening model can influence outcomes across multiple companies.

The authors described the pattern as an “algorithmic monoculture,” borrowing a term commonly used to describe overreliance on a single system. As employers adopt the same AI tools, a candidate’s opportunities can be shaped by a shared algorithm before a human recruiter reviews an application.

The findings arrive amid growing regulatory attention on automated hiring systems. In 2021, the Equal Employment Opportunity Commission launched an initiative focused on artificial intelligence and algorithmic fairness, citing concerns about how automated tools may affect hiring and other workplace decisions.

The agency has warned that employers may be held responsible when algorithmic screening systems create discriminatory outcomes, even when the technology is developed or administered by outside vendors. That position places responsibility on companies using hiring software to monitor whether automated systems disproportionately screen out protected groups.
Sources

For Asian Americans, who make up about 7.4% of the U.S. population but a larger share of workers in technology, engineering and other professional fields, the findings raise questions about how increasingly automated hiring systems may shape access to job opportunities. As more employers adopt similar screening tools, disparities embedded in a single model could affect candidates across multiple companies before a recruiter reviews their applications.