Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring

TL;DR

This study investigates how large language models generate candidate summaries with subtle, name-conditioned evaluative biases that can destabilize fairness in automated hiring assessments.

Contribution

It reveals that while factual resume content remains stable, evaluative language varies with names, especially in open-source models, affecting fairness in LLM-based hiring.

Findings

Evaluative language shows name-conditioned variation, especially at distribution extremes.

Factual resume content remains largely stable across name perturbations.

Open-source models exhibit more pronounced evaluative bias than proprietary models.

Abstract

Research has documented LLMs' name-based bias in hiring and salary recommendations. In this paper, we instead consider a setting where LLMs generate candidate summaries for downstream assessment. In a large-scale controlled study, we analyze nearly one million resume summaries produced by 4 models under systematic race-gender name perturbations, using synthetic resumes and real-world job postings. By decomposing each summary into resume-grounded factual content and evaluative framing, we find that factual content remains largely stable, while evaluative language exhibits subtle name-conditioned variation concentrated in the extremes of the distribution, especially in open-source models. Our hiring simulation demonstrates how evaluative summary transforms directional harm into symmetric instability that might evade conventional fairness audit, highlighting a potential pathway for LLM-to-LLM automation bias.