From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test

TL;DR

This paper introduces a new method to evaluate and reduce cultural bias in large language models by extending the word association test and embedding cultural semantics directly into the models.

Contribution

It presents CultureSteer, a novel approach that embeds cultural-specific semantic associations within LLMs to improve cross-cultural alignment and reduce bias.

Findings

Current LLMs show significant Western bias in word associations.

CultureSteer effectively aligns LLMs with diverse cultural semantics.

Improved cultural sensitivity in downstream tasks.

Abstract

The human-centered word association test (WAT) serves as a cognitive proxy, revealing sociocultural variations through culturally shared semantic expectations and implicit linguistic patterns shaped by lived experiences. We extend this test into an LLM-adaptive, free-relation task to assess the alignment of large language models (LLMs) with cross-cultural cognition. To address culture preference, we propose CultureSteer, an innovative approach that moves beyond superficial cultural prompting by embedding cultural-specific semantic associations directly within the model's internal representation space. Experiments show that current LLMs exhibit significant bias toward Western (notably American) schemas at the word association level. In contrast, our model substantially improves cross-cultural alignment, capturing diverse semantic associations. Further validation on culture-sensitive downstream tasks confirms its efficacy in fostering cognitive alignment across cultures. This work contributes a novel methodological paradigm for enhancing cultural awareness in LLMs, advancing the development of more inclusive language technologies.