Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models

TL;DR

This study systematically examines cultural biases in large language models used for hiring evaluations, revealing cross-cultural disparities and emphasizing the importance of culturally sensitive AI design.

Contribution

It provides the first comprehensive analysis of how LLMs assess interview transcripts across cultures and identities, highlighting linguistic and social biases in AI hiring tools.

Findings

Indian transcripts scored lower than UK ones, even when anonymized.

Name-based substitutions did not significantly affect LLM scores.

Linguistic features like sentence complexity influence evaluations.

Abstract

Artificial Intelligence (AI) is increasingly used in hiring, with large language models (LLMs) having the potential to influence or even make hiring decisions. However, this raises pressing concerns about bias, fairness, and trust, particularly across diverse cultural contexts. Despite their growing role, few studies have systematically examined the potential biases in AI-driven hiring evaluation across cultures. In this study, we conduct a systematic analysis of how LLMs assess job interviews across cultural and identity dimensions. Using two datasets of interview transcripts, 100 from UK and 100 from Indian job seekers, we first examine cross-cultural differences in LLM-generated scores for hirability and related traits. Indian transcripts receive consistently lower scores than UK transcripts, even when they were anonymized, with disparities linked to linguistic features such as sentence complexity and lexical diversity. We then perform controlled identity substitutions (varying names by gender, caste, and region) within the Indian dataset to test for name-based bias. These substitutions do not yield statistically significant effects, indicating that names alone, when isolated from other contextual signals, may not influence LLM evaluations. Our findings underscore the importance of evaluating both linguistic and social dimensions in LLM-driven evaluations and highlight the need for culturally sensitive design and accountability in AI-assisted hiring.