Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMs

TL;DR

This paper investigates how large language models encode and reflect political and social biases related to gendered language reform, revealing biases and inconsistencies in their outputs that mirror ideological stances.

Contribution

It uncovers political biases and internal inconsistencies in LLMs' language use regarding gendered language, highlighting implications for value alignment and model interpretability.

Findings

LLMs align with conservative language preferences when prompted as 'correct' or 'natural'

LLMs show more gender-neutral language use with explicit context

Language ideologies in LLM outputs vary based on context and prompt

Abstract

We study language ideologies in text produced by LLMs through a case study on English gendered language reform (related to role nouns like congressperson/-woman/-man, and singular they). First, we find political bias: when asked to use language that is "correct" or "natural", LLMs use language most similarly to when asked to align with conservative (vs. progressive) values. This shows how LLMs' metalinguistic preferences can implicitly communicate the language ideologies of a particular political group, even in seemingly non-political contexts. Second, we find LLMs exhibit internal inconsistency: LLMs use gender-neutral variants more often when more explicit metalinguistic context is provided. This shows how the language ideologies expressed in text produced by LLMs can vary, which may be unexpected to users. We discuss the broader implications of these findings for value alignment.