Skin-Deep Bias: How Avatar Appearances Shape Perceptions of AI Hiring

TL;DR

This study investigates how avatar appearance cues in AI interviews influence perceptions of fairness and bias, revealing that racial mismatch increases bias perception and partial identity match reduces fairness judgments.

Contribution

It demonstrates that avatar phenotypic traits significantly affect justice attributions in AI interviews, extending social-actor paradigms to AI fairness evaluations.

Findings

Racial mismatch increases perceptions of ethnic bias.

Partial identity match reduces fairness judgments.

Avatar appearance influences trust and bias perceptions.

Abstract

Artificial intelligence is increasingly used in hiring, raising concerns about how applicants perceive these systems. While prior work on algorithmic fairness has emphasized technical bias mitigation, little is known about how avatar identity cues influence applicants' justice attributions in an interview context. We conducted a crowdsourcing study with 215 participants who completed an interview with photorealistic AI avatars varied in phenotypic traits (race and sex), followed by a standardized rejection. Using self-reports, sentiment analysis, and eye tracking, we measured perceptions of trust, fairness, and bias. Results show that racial mismatch heightened perceptions of ethnic bias, while partial match (sharing only one identity) reduced fairness judgments compared to both full and no match. This work extends the Computers-Are-Social-Actors paradigm by demonstrating that avatar appearances shape justicerelated evaluations of AI. We contribute to HCI by revealing how identity cues influence fairness attributions and offer actionable insights for designing equitable AI interview systems.