Nationality, Race, and Ethnicity Biases in and Consequences of Detecting AI-Generated Self-Presentations

TL;DR

This study examines how race, ethnicity, and nationality influence perceptions of AI-generated self-presentations in college applications, revealing biases and their impact on applicant evaluations.

Contribution

It demonstrates the influence of content and source cues, including racial and national stereotypes, on AI detection and applicant perception in high-stakes settings.

Findings

Content heuristics dominate AI detection.

Nationality cues affect AI attribution, especially for international students.

Racial stereotypes influence AI judgments, impacting applicant evaluations.

Abstract

This study builds on person perception and human AI interaction (HAII) theories to investigate how content and source cues, specifically race, ethnicity, and nationality, affect judgments of AI-generated content in a high-stakes self-presentation context: college applications. Results of a pre-registered experiment with a nationally representative U.S. sample (N = 644) show that content heuristics, such as linguistic style, played a dominant role in AI detection. Source heuristics, such as nationality, also emerged as a significant factor, with international students more likely to be perceived as using AI, especially when their statements included AI-sounding features. Interestingly, Asian and Hispanic applicants were more likely to be judged as AI users when labeled as domestic students, suggesting interactions between racial stereotypes and AI detection. AI attribution led to lower perceptions of personal statement quality and authenticity, as well as negative evaluations of the applicant's competence, sociability, morality, and future success.