Large Language Models as Mirrors of Societal Moral Standards

TL;DR

This paper investigates how large language models reflect societal moral standards across cultures, revealing biases and limitations in capturing moral nuances, and emphasizes the need for culturally aware AI systems.

Contribution

It replicates prior findings on moral norm representation in language models and evaluates their cross-cultural accuracy using large survey datasets.

Findings

Biases exist in models across cultures

Models poorly capture cultural moral nuances

BLOOM performs best among tested models

Abstract

Prior research has demonstrated that language models can, to a limited extent, represent moral norms in a variety of cultural contexts. This research aims to replicate these findings and further explore their validity, concentrating on issues like 'homosexuality' and 'divorce'. This study evaluates the effectiveness of these models using information from two surveys, the WVS and the PEW, that encompass moral perspectives from over 40 countries. The results show that biases exist in both monolingual and multilingual models, and they typically fall short of accurately capturing the moral intricacies of diverse cultures. However, the BLOOM model shows the best performance, exhibiting some positive correlations, but still does not achieve a comprehensive moral understanding. This research underscores the limitations of current PLMs in processing cross-cultural differences in values and highlights the importance of developing culturally aware AI systems that better align with universal human values.