Divine LLaMAs: Bias, Stereotypes, Stigmatization, and Emotion Representation of Religion in Large Language Models

TL;DR

This paper investigates how large language models represent different religions through emotion attribution, revealing biases, stereotypes, and stigmatization, and highlights the need to address these issues to prevent perpetuating harmful perceptions.

Contribution

It introduces a novel analysis of religious representation in LLMs using emotion attribution, uncovering cultural biases and stereotypes that influence model outputs.

Findings

Major Western religions are represented with nuanced understanding.

Eastern religions like Hinduism and Buddhism are strongly stereotyped.

Judaism and Islam face stigmatization and model refusal.

Abstract

Emotions play important epistemological and cognitive roles in our lives, revealing our values and guiding our actions. Previous work has shown that LLMs display biases in emotion attribution along gender lines. However, unlike gender, which says little about our values, religion, as a socio-cultural system, prescribes a set of beliefs and values for its followers. Religions, therefore, cultivate certain emotions. Moreover, these rules are explicitly laid out and interpreted by religious leaders. Using emotion attribution, we explore how different religions are represented in LLMs. We find that: Major religions in the US and European countries are represented with more nuance, displaying a more shaded model of their beliefs. Eastern religions like Hinduism and Buddhism are strongly stereotyped. Judaism and Islam are stigmatized -- the models' refusal skyrocket. We ascribe these to cultural bias in LLMs and the scarcity of NLP literature on religion. In the rare instances where religion is discussed, it is often in the context of toxic language, perpetuating the perception of these religions as inherently toxic. This finding underscores the urgent need to address and rectify these biases. Our research underscores the crucial role emotions play in our lives and how our values influence them.