What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring

TL;DR

This paper investigates how representation criteria, like gender-balanced candidate displays, influence human gender bias in hiring decisions, revealing that such strategies can mitigate bias in some contexts but not all.

Contribution

It introduces a controlled experimental platform to decouple world distribution from human bias and evaluates the effectiveness of representation criteria as bias mitigation strategies.

Findings

Balancing gender representation can reduce bias in some professions.

Gender of decision-maker influences hiring bias.

Task complexity and candidate slate composition affect decisions.

Abstract

Although systematic biases in decision-making are widely documented, the ways in which they emerge from different sources is less understood. We present a controlled experimental platform to study gender bias in hiring by decoupling the effect of world distribution (the gender breakdown of candidates in a specific profession) from bias in human decision-making. We explore the effectiveness of \textit{representation criteria}, fixed proportional display of candidates, as an intervention strategy for mitigation of gender bias by conducting experiments measuring human decision-makers' rankings for who they would recommend as potential hires. Experiments across professions with varying gender proportions show that balancing gender representation in candidate slates can correct biases for some professions where the world distribution is skewed, although doing so has no impact on other professions where human persistent preferences are at play. We show that the gender of the decision-maker, complexity of the decision-making task and over- and under-representation of genders in the candidate slate can all impact the final decision. By decoupling sources of bias, we can better isolate strategies for bias mitigation in human-in-the-loop systems.