Analyzing values about gendered language reform in LLMs' revisions

TL;DR

This paper investigates how large language models revise gendered role nouns and their justifications, assessing alignment with feminist and trans-inclusive language reforms, and explores their sensitivity to contextual effects in language use.

Contribution

It provides a novel analysis of LLMs' handling of gendered language reforms and their contextual sensitivity, informed by sociolinguistic insights.

Findings

LLMs often align with feminist and trans-inclusive language reforms.

LLMs show sensitivity to contextual effects similar to humans.

Implications for improving value alignment in language models.

Abstract

Within the common LLM use case of text revision, we study LLMs' revision of gendered role nouns (e.g., outdoorsperson/woman/man) and their justifications of such revisions. We evaluate their alignment with feminist and trans-inclusive language reforms for English. Drawing on insight from sociolinguistics, we further assess if LLMs are sensitive to the same contextual effects in the application of such reforms as people are, finding broad evidence of such effects. We discuss implications for value alignment.