Measuring Spiritual Values and Bias of Large Language Models

TL;DR

This paper investigates the spiritual values and biases present in large language models, revealing diverse spiritual perspectives and their influence on social fairness tasks, and proposes pre-training on spiritual texts to mitigate bias.

Contribution

It introduces a method to evaluate spiritual biases in LLMs and demonstrates how targeted pre-training can reduce such biases.

Findings

LLMs exhibit diverse spiritual values contrary to stereotypes.

Spiritual values influence LLMs' sensitivity in hate speech detection.

Pre-training on spiritual texts helps mitigate spiritual bias.

Abstract

Large language models (LLMs) have become integral tool for users from various backgrounds. LLMs, trained on vast corpora, reflect the linguistic and cultural nuances embedded in their pre-training data. However, the values and perspectives inherent in this data can influence the behavior of LLMs, leading to potential biases. As a result, the use of LLMs in contexts involving spiritual or moral values necessitates careful consideration of these underlying biases. Our work starts with verification of our hypothesis by testing the spiritual values of popular LLMs. Experimental results show that LLMs' spiritual values are quite diverse, as opposed to the stereotype of atheists or secularists. We then investigate how different spiritual values affect LLMs in social-fairness scenarios e.g., hate speech identification). Our findings reveal that different spiritual values indeed lead to different sensitivity to different hate target groups. Furthermore, we propose to continue pre-training LLMs on spiritual texts, and empirical results demonstrate the effectiveness of this approach in mitigating spiritual bias.