When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writing

TL;DR

This study investigates how gender stereotypes in AI text suggestions influence human writing, revealing that while anti-stereotypical prompts can reduce bias, stereotypical narratives still prevail, highlighting limited debiasing effectiveness.

Contribution

It provides empirical evidence on the influence of stereotype-laden AI suggestions on human co-writing and assesses the partial effectiveness of technical debiasing methods.

Findings

Anti-stereotypical suggestions increase anti-stereotypical stories

Pro-stereotypical narratives still dominate co-written stories

Technical debiasing only partially reduces gender bias

Abstract

AI-based systems such as language models have been shown to replicate and even amplify social biases reflected in their training data. Among other questionable behaviors, this can lead to AI-generated text--and text suggestions--that contain normatively inappropriate stereotypical associations. Little is known, however, about how this behavior impacts the writing produced by people using these systems. We address this gap by measuring how much impact stereotypes or anti-stereotypes in English single-word LM predictive text suggestions have on the stories that people write using those tools in a co-writing scenario. We find that ($n=414$), LM suggestions that challenge stereotypes sometimes lead to a significantly increased rate of anti-stereotypical co-written stories. However, despite this increased rate of anti-stereotypical stories, pro-stereotypical narratives still dominated the co-written stories, demonstrating that technical debiasing is only a partially effective strategy to alleviate harms from human-AI collaboration.