Language Models Entangle Language and Culture

TL;DR

This paper investigates how language choice affects the quality and cultural context of responses generated by large language models, revealing disparities especially in low-resource languages and highlighting entanglement of language and culture.

Contribution

The study introduces real-world questions and a cultural benchmark to evaluate language and cultural entanglement in LLMs, highlighting disparities across languages.

Findings

LLMs produce lower quality responses in low-resource languages.

Language influences the cultural context in generated responses.

Cultural context impacts answer quality.

Abstract

Users should not be systemically disadvantaged by the language they use for interacting with LLMs; i.e. users across languages should get responses of similar quality irrespective of language used. In this work, we create a set of real-world open-ended questions based on our analysis of the WildChat dataset and use it to evaluate whether responses vary by language, specifically, whether answer quality depends on the language used to query the model. We also investigate how language and culture are entangled in LLMs such that choice of language changes the cultural information and context used in the response by using LLM-as-a-Judge to identify the cultural context present in responses. To further investigate this, we evaluate LLMs on a translated subset of the CulturalBench benchmark across multiple languages. Our evaluations reveal that LLMs consistently provide lower quality answers to open-ended questions in low resource languages. We find that language significantly impacts the cultural context used by the model. This difference in context impacts the quality of the downstream answer.