Mind the Gap: Pitfalls of LLM Alignment with Asian Public Opinion

TL;DR

This paper conducts a multilingual audit of large language models to assess their cultural alignment, especially regarding religious viewpoints in Asian societies, revealing significant gaps and biases that require targeted interventions.

Contribution

It provides a comprehensive multilingual analysis of LLMs' cultural alignment focusing on religion across Asian regions, highlighting persistent biases and the limitations of lightweight mitigation strategies.

Findings

Models often misrepresent minority religious viewpoints.

Lightweight interventions only partially reduce cultural gaps.

Bias benchmarks reveal ongoing harms and under-representation.

Abstract

Large Language Models (LLMs) are increasingly being deployed in multilingual, multicultural settings, yet their reliance on predominantly English-centric training data risks misalignment with the diverse cultural values of different societies. In this paper, we present a comprehensive, multilingual audit of the cultural alignment of contemporary LLMs including GPT-4o-Mini, Gemini-2.5-Flash, Llama 3.2, Mistral and Gemma 3 across India, East Asia and Southeast Asia. Our study specifically focuses on the sensitive domain of religion as the prism for broader alignment. To facilitate this, we conduct a multi-faceted analysis of every LLM's internal representations, using log-probs/logits, to compare the model's opinion distributions against ground-truth public attitudes. We find that while the popular models generally align with public opinion on broad social issues, they consistently fail to accurately represent religious viewpoints, especially those of minority groups, often amplifying negative stereotypes. Lightweight interventions, such as demographic priming and native language prompting, partially mitigate but do not eliminate these cultural gaps. We further show that downstream evaluations on bias benchmarks (such as CrowS-Pairs, IndiBias, ThaiCLI, KoBBQ) reveal persistent harms and under-representation in sensitive contexts. Our findings underscore the urgent need for systematic, regionally grounded audits to ensure equitable global deployment of LLMs.