Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency

TL;DR

This paper introduces a self-supervised framework that enhances cross-lingual cultural knowledge alignment in LLMs, reducing Western bias and improving performance across languages by leveraging multilingual self-consistency and self-critique.

Contribution

It proposes a novel multilingual self-consistency approach to surface and transfer cultural knowledge across languages in LLMs, improving cultural alignment without external data.

Findings

Improves cultural alignment performance on English queries by 5.03% on BLEnD benchmark.

Leverages multilingual self-consistency to identify reliable responses across languages.

Uses a self-critique mechanism to transfer cultural knowledge to weaker languages.

Abstract

Although Large Language Models (LLMs) demonstrate strong capabilities across various tasks, they exhibit significant performance discrepancies across languages. While prompting LLMs in English typically yields the highest general performance, it often induces a Western-centric bias, hindering the model's ability to accurately reflect diverse cultural knowledge. We hypothesize that LLMs already possess rich cultural knowledge embedded within local-language representations, but fail to retrieve it when prompted in English. To bridge this cross-lingual knowledge gap, we propose a novel self-supervised framework. Our method leverages multilingual self-consistency to identify the most reliable cultural responses across languages, combined with a self-critique mechanism to transfer this knowledge to the weaker language. Evaluations on the BLEnD benchmark demonstrate that our approach significantly improves cultural alignment-boosting performance on English queries by an average of 5.03%-relying entirely on self-generated data. Ultimately, our work demonstrates that latent cultural knowledge can be successfully surfaced and propagated across languages, enabling more culturally equitable and consistent LLMs.