Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study

TL;DR

This study evaluates ChatGPT's ability to adapt to different cultural contexts, revealing strong American cultural alignment and highlighting the influence of prompt language on cultural bias in responses.

Contribution

It provides an empirical analysis of ChatGPT's cultural responsiveness and the impact of prompt language on cultural bias, emphasizing the need for greater cultural diversity in AI training.

Findings

ChatGPT aligns strongly with American culture when prompted in English.

Responses are less culturally diverse when using English prompts.

Cultural differences are less pronounced in responses to English prompts.

Abstract

The recent release of ChatGPT has garnered widespread recognition for its exceptional ability to generate human-like responses in dialogue. Given its usage by users from various nations and its training on a vast multilingual corpus that incorporates diverse cultural and societal norms, it is crucial to evaluate its effectiveness in cultural adaptation. In this paper, we investigate the underlying cultural background of ChatGPT by analyzing its responses to questions designed to quantify human cultural differences. Our findings suggest that, when prompted with American context, ChatGPT exhibits a strong alignment with American culture, but it adapts less effectively to other cultural contexts. Furthermore, by using different prompts to probe the model, we show that English prompts reduce the variance in model responses, flattening out cultural differences and biasing them towards American culture. This study provides valuable insights into the cultural implications of ChatGPT and highlights the necessity of greater diversity and cultural awareness in language technologies.