Irrelevant Alternatives Bias Large Language Model Hiring Decisions

TL;DR

This paper demonstrates that large language models like GPT-3.5 and GPT-4 exhibit the attraction effect, a human cognitive bias, in hiring decisions, which can be amplified by irrelevant attributes such as gender.

Contribution

It provides empirical evidence that LLMs display the attraction effect in hiring scenarios, highlighting biases similar to human cognitive biases.

Findings

GPT-4 shows greater bias variation than GPT-3.5.

Irrelevant attributes like gender amplify the bias.

Bias persists despite warnings and role variations.

Abstract

We investigate whether LLMs display a well-known human cognitive bias, the attraction effect, in hiring decisions. The attraction effect occurs when the presence of an inferior candidate makes a superior candidate more appealing, increasing the likelihood of the superior candidate being chosen over a non-dominated competitor. Our study finds consistent and significant evidence of the attraction effect in GPT-3.5 and GPT-4 when they assume the role of a recruiter. Irrelevant attributes of the decoy, such as its gender, further amplify the observed bias. GPT-4 exhibits greater bias variation than GPT-3.5. Our findings remain robust even when warnings against the decoy effect are included and the recruiter role definition is varied.