Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models

TL;DR

This study evaluates how large language models reflect WEIRD cultural biases and their alignment with human rights, revealing a trade-off between cultural diversity and potential human rights violations in model outputs.

Contribution

It provides a comparative analysis of multiple LLMs' cultural biases and their implications for human rights, highlighting the need for embedding ethical principles in AI models.

Findings

Models with less WEIRD alignment show more cultural diversity.

Lower WEIRD alignment correlates with increased human rights violations.

Some models endorse harmful gender norms.

Abstract

Large language models (LLMs) are often trained on data that reflect WEIRD values: Western, Educated, Industrialized, Rich, and Democratic. This raises concerns about cultural bias and fairness. Using responses to the World Values Survey, we evaluated five widely used LLMs: GPT-3.5, GPT-4, Llama-3, BLOOM, and Qwen. We measured how closely these responses aligned with the values of the WEIRD countries and whether they conflicted with human rights principles. To reflect global diversity, we compared the results with the Universal Declaration of Human Rights and three regional charters from Asia, the Middle East, and Africa. Models with lower alignment to WEIRD values, such as BLOOM and Qwen, produced more culturally varied responses but were 2% to 4% more likely to generate outputs that violated human rights, especially regarding gender and equality. For example, some models agreed with the statements ``a man who cannot father children is not a real man'' and ``a husband should always know where his wife is'', reflecting harmful gender norms. These findings suggest that as cultural representation in LLMs increases, so does the risk of reproducing discriminatory beliefs. Approaches such as Constitutional AI, which could embed human rights principles into model behavior, may only partly help resolve this tension.