Attributing Culture-Conditioned Generations to Pretraining Corpora

TL;DR

This paper introduces the MEMOed framework to analyze how large language models' culture-conditioned generations are influenced by pretraining data, revealing biases toward frequent cultures and entities.

Contribution

The paper presents a novel MEMOed framework for attributing culture-conditioned generations to memorization from pretraining data, highlighting biases in language models.

Findings

High-frequency cultures in pretraining data lead to more memorized generations.

Some low-frequency cultures produce no memorized generations.

Models favor generating high-frequency entities regardless of culture conditioning.

Abstract

In open-ended generative tasks like narrative writing or dialogue, large language models often exhibit cultural biases, showing limited knowledge and generating templated outputs for less prevalent cultures. Recent works show that these biases may stem from uneven cultural representation in pretraining corpora. This work investigates how pretraining leads to biased culture-conditioned generations by analyzing how models associate entities with cultures based on pretraining data patterns. We propose the MEMOed framework (MEMOrization from pretraining document) to determine whether a generation for a culture arises from memorization. Using MEMOed on culture-conditioned generations about food and clothing for 110 cultures, we find that high-frequency cultures in pretraining data yield more generations with memorized symbols, while some low-frequency cultures produce none. Additionally, the model favors generating entities with extraordinarily high frequency regardless of the conditioned culture, reflecting biases toward frequent pretraining terms irrespective of relevance. We hope that the MEMOed framework and our insights will inspire more works on attributing model performance on pretraining data.