FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes

TL;DR

This paper introduces INDIC-BIAS, a comprehensive Indian-centric benchmark to evaluate fairness in large language models across diverse identities, revealing prevalent biases and stereotypes in current models and emphasizing the need for culturally sensitive AI.

Contribution

The paper presents the first India-specific fairness benchmark, INDIC-BIAS, with a large set of culturally relevant scenarios to assess biases in LLMs, filling a significant research gap.

Findings

Models exhibit strong negative biases against marginalized Indian identities.

Models often reinforce stereotypes even when prompted to rationalize decisions.

Current LLMs struggle to mitigate biases in culturally diverse contexts.

Abstract

Existing studies on fairness are largely Western-focused, making them inadequate for culturally diverse countries such as India. To address this gap, we introduce INDIC-BIAS, a comprehensive India-centric benchmark designed to evaluate fairness of LLMs across 85 identity groups encompassing diverse castes, religions, regions, and tribes. We first consult domain experts to curate over 1,800 socio-cultural topics spanning behaviors and situations, where biases and stereotypes are likely to emerge. Grounded in these topics, we generate and manually validate 20,000 real-world scenario templates to probe LLMs for fairness. We structure these templates into three evaluation tasks: plausibility, judgment, and generation. Our evaluation of 14 popular LLMs on these tasks reveals strong negative biases against marginalized identities, with models frequently reinforcing common stereotypes. Additionally, we find that models struggle to mitigate bias even when explicitly asked to rationalize their decision. Our evaluation provides evidence of both allocative and representational harms that current LLMs could cause towards Indian identities, calling for a more cautious usage in practical applications. We release INDIC-BIAS as an open-source benchmark to advance research on benchmarking and mitigating biases and stereotypes in the Indian context.