Investigations of Performance and Bias in Human-AI Teamwork in Hiring

TL;DR

This study investigates how AI model performance and bias influence human decision-making in hiring, revealing that high-performance models can improve outcomes but also affect bias and conformity in complex ways.

Contribution

It provides a large-scale empirical analysis of how different NLP classifiers impact human-AI teamwork and bias transfer in hiring decisions.

Findings

High-performance models improve human accuracy in candidate selection.

Some models reduce bias, others increase it.

Model architecture influences human-AI conformity and bias.

Abstract

In AI-assisted decision-making, effective hybrid (human-AI) teamwork is not solely dependent on AI performance alone, but also on its impact on human decision-making. While prior work studies the effects of model accuracy on humans, we endeavour here to investigate the complex dynamics of how both a model's predictive performance and bias may transfer to humans in a recommendation-aided decision task. We consider the domain of ML-assisted hiring, where humans -- operating in a constrained selection setting -- can choose whether they wish to utilize a trained model's inferences to help select candidates from written biographies. We conduct a large-scale user study leveraging a re-created dataset of real bios from prior work, where humans predict the ground truth occupation of given candidates with and without the help of three different NLP classifiers (random, bag-of-words, and deep neural network). Our results demonstrate that while high-performance models significantly improve human performance in a hybrid setting, some models mitigate hybrid bias while others accentuate it. We examine these findings through the lens of decision conformity and observe that our model architecture choices have an impact on human-AI conformity and bias, motivating the explicit need to assess these complex dynamics prior to deployment.