ALIGN: Word Association Learning for Cultural Alignment in Large Language Models

TL;DR

This paper presents a cost-effective method to improve cultural alignment in large language models by fine-tuning them on native speakers' word-association norms, leading to significant cultural value shifts.

Contribution

Introducing a cognitively grounded fine-tuning approach using word association data to enhance cultural alignment in large language models.

Findings

Significant lexical alignment improvements (16-20% English, 43-165% Mandarin).

High-level cultural value shifts observed, especially in diverging questions.

Models with 7-8B parameters match or outperform larger baselines.

Abstract

Large language models (LLMs) exhibit cultural bias from overrepresented viewpoints in training data, yet cultural alignment remains a challenge due to limited cultural knowledge and a lack of exploration into effective learning approaches. We introduce a cost-efficient and cognitively grounded method: fine-tuning LLMs on native speakers' word-association norms, leveraging cognitive psychology findings that such associations capture cultural knowledge. Using word association datasets from native speakers in the US (English) and China (Mandarin), we train Llama-3.1-8B and Qwen-2.5-7B via supervised fine-tuning and preference optimization. We evaluate models' cultural alignment through a two-tier evaluation framework that spans lexical associations and cultural value alignment using the World Values Survey. Results show significant improvements in lexical alignment (16-20% English, 43-165% Mandarin on Precision@5) and high-level cultural value shifts. On a subset of 50 questions where US and Chinese respondents diverge most, fine-tuned Qwen nearly doubles its response alignment with Chinese values (13 to 25). Remarkably, our trained 7-8B models match or exceed vanilla 70B baselines, demonstrating that a few million of culture-grounded associations achieve value alignment without expensive retraining. Our work highlights both the promise and the need for future research grounded in human cognition in improving cultural alignment in AI models.