Isolating Culture Neurons in Multilingual Large Language Models

TL;DR

This paper investigates how multilingual large language models encode cultural information, identifying culture-specific neurons that can be isolated and modulated independently of language neurons, with implications for fairness and alignment.

Contribution

It introduces a methodology and dataset to localize and isolate culture-specific neurons in multilingual LLMs, showing their distinct encoding and potential for targeted editing.

Findings

Culture is encoded in distinct neuron populations in LLMs.

Culture neurons are mainly located in upper layers.

Culture neurons can be modulated independently of language neurons.

Abstract

Language and culture are deeply intertwined, yet it has been unclear how and where multilingual large language models encode culture. Here, we build on an established methodology for identifying language-specific neurons to localize and isolate culture-specific neurons, carefully disentangling their overlap and interaction with language-specific neurons. To facilitate our experiments, we introduce MUREL, a curated dataset of 85.2 million tokens spanning six different cultures. Our localization and intervention experiments show that LLMs encode different cultures in distinct neuron populations, predominantly in upper layers, and that these culture neurons can be modulated largely independently of language-specific neurons or those specific to other cultures. These findings suggest that cultural knowledge and propensities in multilingual language models can be selectively isolated and edited, with implications for fairness, inclusivity, and alignment. Code and data are available at https://github.com/namazifard/Culture_Neurons.