StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

TL;DR

StereoTales introduces a multilingual dataset and evaluation pipeline to systematically study social biases in open-ended LLM generation across 10 languages and 79 socio-demographic attributes.

Contribution

It provides the first large-scale multilingual dataset with annotations and a pipeline for analyzing social bias and harmful stereotypes in LLMs.

Findings

All evaluated models emit harmful stereotypes regardless of size or capabilities.

Prompt language influences the stereotypes that appear, amplifying biases against locally salient groups.

Human and LLM harmfulness judgments are broadly aligned, with some disagreements on specific attributes.

Abstract

Multilingual studies of social bias in open-ended LLM generation remain limited: most existing benchmarks are English-centric, template-based, or restricted to recognizing pre-specified stereotypes. We introduce StereoTales, a multilingual dataset and evaluation pipeline for systematically studying the emergence of social bias in open-ended LLM generation. The dataset covers 10 languages and 79 socio-demographic attributes, and comprises over 650k stories generated by 23 recent LLMs, each annotated with the socio-demographic profile of the protagonist across 19 dimensions. From these, we apply statistical tests to identify more than 1{,}500 over-represented associations, which we then rate for harmfulness through both a panel of humans (N = 247) and the same LLMs. We report three main findings. \textbf{(i)} Every model we evaluate emits consequential harmful stereotypes in open-ended generation, regardless of size or capabilities, and these associations are largely shared across providers rather than isolated misbehaviors. \textbf{(ii)} Prompt language strongly shapes which stereotypes appear: rather than transferring as a shared set of biases, harmful associations adapt culturally to the prompt language and amplify bias against locally salient protected groups. \textbf{(iii)} Human and LLM harmfulness judgments are broadly aligned (Spearman $\rho=0.62$), with disagreements concentrating on specific attribute classes rather than specific providers. To support further analyses, we release the evaluation code and the dataset, including model generations, attribute annotations, and harmfulness ratings.