Sima AIunty: Caste Audit in LLM-Driven Matchmaking

TL;DR

This study audits caste bias in large language models used for matchmaking, revealing that models tend to favor same-caste matches and reproduce traditional caste hierarchies, raising concerns about reinforcing social exclusion.

Contribution

It provides a controlled analysis of caste bias across multiple LLMs in a sensitive social domain, highlighting the reproduction of caste hierarchies in AI evaluations.

Findings

Same-caste matches rated up to 25% higher than inter-caste matches.

Models reproduce traditional caste hierarchies in matchmaking evaluations.

Caste bias persists across different LLM families and social dimensions.

Abstract

Social and personal decisions in relational domains such as matchmaking are deeply entwined with cultural norms and historical hierarchies, and can potentially be shaped by algorithmic and AI-mediated assessments of compatibility, acceptance, and stability. In South Asian contexts, caste remains a central aspect of marital decision-making, yet little is known about how contemporary large language models (LLMs) reproduce or disrupt caste-based stratification in such settings. In this work, we conduct a controlled audit of caste bias in LLM-mediated matchmaking evaluations using real-world matrimonial profiles. We vary caste identity across Brahmin, Kshatriya, Vaishya, Shudra, and Dalit, and income across five buckets, and evaluate five LLM families (GPT, Gemini, Llama, Qwen, and BharatGPT). Models are prompted to assess profiles along dimensions of social acceptance, marital stability, and cultural compatibility. Our analysis reveals consistent hierarchical patterns across models: same-caste matches are rated most favorably, with average ratings up to 25% higher (on a 10-point scale) than inter-caste matches, which are further ordered according to traditional caste hierarchy. These findings highlight how existing caste hierarchies are reproduced in LLM decision-making and underscore the need for culturally grounded evaluation and intervention strategies in AI systems deployed in socially sensitive domains, where such systems risk reinforcing historical forms of exclusion.