Human, Algorithm, or Both? Gender Bias in Human-Augmented Recruiting

TL;DR

This study empirically compares gender bias in human, AI, and hybrid recruiting, finding that human and combined approaches produce fairer outcomes than AI alone, emphasizing human oversight's role in mitigating bias.

Contribution

It provides one of the first empirical analyses comparing gender bias across human, AI, and hybrid recruitment methods, highlighting the benefits of human-AI collaboration.

Findings

Humans produce fairer gender outcomes than AI alone.

Combining human and AI methods yields the fairest candidate lists.

Human oversight significantly reduces gender bias in recruiting.

Abstract

Recent years have seen rapid growth in the market for HR technology and AI-driven HR solutions in particular. This popularity has also resulted in increased attention to the negative aspects of using AI to support hiring practices, such as the risk of reinforcing existing biases against vulnerable groups based on gender or other sensitive attributes. Combining human experience with AI efficiency in making recruiting and selection decisions has the potential to help mitigate these biases, but despite a considerable amount of research on fairness in algorithmic hiring, actual empirical evaluations comparing the fairness of human, AI, and human-augmented decision-making remain scarce. In this study, we address this gap by presenting a quantitative analysis of gender bias across three scenarios of a real-world recruitment platform: (1) recruiters searching a CV database manually for relevant candidates, (2) AI-driven matching between candidates and jobs, and (3) a combination of human and AI-driven recruiting. We find that human recruiters produce lists of candidates that are fairer in terms of gender than the AI-only solution, with more deliberation by humans resulting in fairer outcomes. However, the combination of human and AI-driven is more than the sum of its parts and produces the fairest candidate lists: interacting with the slate of recommended candidates first before manually searching for additional candidates has a beneficial effect on the gender fairness of the set of candidates that are viewed, clicked, and contacted afterwards. Our work provides one of the first empirical comparisons of fairness across human, AI, and hybrid recruiting processes, offering evidence to inform the development of more equitable hiring practices and highlighting the importance of human oversight for mitigating bias in algorithmic hiring.