Exploring Changes in Nation Perception with Nationality-Assigned Personas in LLMs

TL;DR

This study investigates how assigning specific nationality personas to large language models influences their perceptions of countries, revealing biases and the importance of fairness mechanisms in AI outputs.

Contribution

It introduces a comprehensive analysis of nationality-based persona assignment effects on LLM perceptions, highlighting biases and the partial alignment with human responses.

Findings

LLMs favor Western European nations across personas.

Nationality personas influence LLMs to treat their own region more favorably.

Evaluations correlate with human surveys but do not fully match human values.

Abstract

Persona assignment has become a common strategy for customizing LLM use to particular tasks and contexts. In this study, we explore how evaluation of different nations change when LLMs are assigned specific nationality personas. We assign 193 different nationality personas (e.g., an American person) to four LLMs and examine how the LLM evaluations (or ''perceptions'')of countries change. We find that all LLM-persona combinations tend to favor Western European nations, though nation-personas push LLM behaviors to focus more on and treat the nation-persona's own region more favorably. Eastern European, Latin American, and African nations are treated more negatively by different nationality personas. We additionally find that evaluations by nation-persona LLMs of other nations correlate with human survey responses but fail to match the values closely. Our study provides insight into how biases and stereotypes are realized within LLMs when adopting different national personas. In line with the ''Blueprint for an AI Bill of Rights'', our findings underscore the critical need for developing mechanisms to ensure that LLM outputs promote fairness and avoid over-generalization.