Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon

TL;DR

This paper investigates how large language models transfer cultural knowledge across languages, revealing an asymmetric transfer pattern influenced by data frequency, with implications for multilingual model development.

Contribution

It introduces an interpretable framework for analyzing cultural transfer in LLMs and uncovers the asymmetric transfer phenomenon linked to data frequency.

Findings

Bidirectional transfer between English and high-resource languages.

Limited reverse transfer from low-resource languages to English.

Frequency of cultural concepts in training data correlates with transfer ease.

Abstract

Despite substantial research efforts evaluating how well large language models~(LLMs) handle global cultural diversity, the mechanisms behind their cultural knowledge acquisition, particularly in multilingual settings, remain unclear. We study this question by investigating how cultural knowledge transfers across languages during language adaptation of LLMs. We introduce an interpretable framework for studying this transfer, ensuring training data transparency and controlling transfer effects. Through a study of four non-Anglophonic cultures, we observe bidirectional cultural transfer between English and other high-resource languages, while low-resource languages primarily transfer knowledge to English with limited reverse flow. To explain this asymmetric phenomenon, we propose a frequency-based hypothesis: cultural knowledge appearing more frequently in the pretraining data transfers more easily, which is supported by empirical analysis of the training corpora.