Surprising gender biases in GPT

TL;DR

This paper uncovers significant gender biases in GPT, showing it associates masculine stereotypes with females and exhibits biased moral judgments, emphasizing the need for careful bias management in AI systems.

Contribution

The study systematically reveals gender biases in GPT's demographic attributions and moral judgments, highlighting implicit biases that are not evident through direct questioning.

Findings

GPT assigns masculine stereotypes to females more often than vice versa.

GPT exhibits biased moral judgments favoring males in high-stakes scenarios.

Biases are implicit and do not appear when directly asked to rank moral violations.

Abstract

We present seven experiments exploring gender biases in GPT. Initially, GPT was asked to generate demographics of a potential writer of twenty phrases containing feminine stereotypes and twenty with masculine stereotypes. Results show a strong asymmetry, with stereotypically masculine sentences attributed to a female more often than vice versa. For example, the sentence "I love playing fotbal! Im practicing with my cosin Michael" was constantly assigned by ChatGPT to a female writer. This phenomenon likely reflects that while initiatives to integrate women in traditionally masculine roles have gained momentum, the reverse movement remains relatively underdeveloped. Subsequent experiments investigate the same issue in high-stakes moral dilemmas. GPT-4 finds it more appropriate to abuse a man to prevent a nuclear apocalypse than to abuse a woman. This bias extends to other forms of violence central to the gender parity debate (abuse), but not to those less central (torture). Moreover, this bias increases in cases of mixed-sex violence for the greater good: GPT-4 agrees with a woman using violence against a man to prevent a nuclear apocalypse but disagrees with a man using violence against a woman for the same purpose. Finally, these biases are implicit, as they do not emerge when GPT-4 is directly asked to rank moral violations. These results highlight the necessity of carefully managing inclusivity efforts to prevent unintended discrimination.