On The Conceptualization and Societal Impact of Cross-Cultural Bias
Vitthal Bhandari

TL;DR
This paper reviews recent literature on cultural bias in NLP, emphasizing the importance of stakeholder engagement and proposing a framework for conceptualizing and evaluating societal impacts of cross-cultural bias in language models.
Contribution
It analyzes 20 recent papers to develop a set of observations for better conceptualizing and assessing the societal impact of cultural bias in NLP models.
Findings
Identified gaps in current bias evaluation methods.
Proposed a framework for societal impact assessment.
Highlighted the need for stakeholder engagement in bias evaluation.
Abstract
Research has shown that while large language models (LLMs) can generate their responses based on cultural context, they are not perfect and tend to generalize across cultures. However, when evaluating the cultural bias of a language technology on any dataset, researchers may choose not to engage with stakeholders actually using that technology in real life, which evades the very fundamental problem they set out to address. Inspired by the work done by arXiv:2005.14050v2, I set out to analyse recent literature about identifying and evaluating cultural bias in Natural Language Processing (NLP). I picked out 20 papers published in 2025 about cultural bias and came up with a set of observations to allow NLP researchers in the future to conceptualize bias concretely and evaluate its harms effectively. My aim is to advocate for a robust assessment of the societal impact of language…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Computational and Text Analysis Methods · Ethics and Social Impacts of AI
