How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns

TL;DR

This paper investigates how language models handle gender-neutral pronouns across Danish, English, and Swedish, revealing a gap between human ease of processing and model performance, which may hinder adoption.

Contribution

It demonstrates that language models exhibit conservativity towards gender-neutral pronouns, highlighting a need for adaptation to support inclusive language use.

Findings

Gender-neutral pronouns have higher perplexity in models.

Models show more dispersed attention with neutral pronouns.

Downstream performance on tasks is worse with neutral pronouns.

Abstract

Gender-neutral pronouns have recently been introduced in many languages to a) include non-binary people and b) as a generic singular. Recent results from psycholinguistics suggest that gender-neutral pronouns (in Swedish) are not associated with human processing difficulties. This, we show, is in sharp contrast with automated processing. We show that gender-neutral pronouns in Danish, English, and Swedish are associated with higher perplexity, more dispersed attention patterns, and worse downstream performance. We argue that such conservativity in language models may limit widespread adoption of gender-neutral pronouns and must therefore be resolved.