Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contexts

TL;DR

This study investigates how large language models generate personas related to Palestinian and Israeli identities in different contexts, revealing biases, fairness considerations, and the influence of prompts on their outputs.

Contribution

It provides a comprehensive analysis of LLM-generated geopolitical personas, highlighting biases, fairness dynamics, and the effects of explicit instructions across multiple models and conditions.

Findings

Palestinian profiles in war contexts often show lower socioeconomic status.

Israeli profiles tend to retain middle-class and professional attributes.

Explicit fairness prompts lead to diverse changes, including increased non-binary gender inferences.

Abstract

Large language models (LLMs) are increasingly utilised for social simulation and persona generation, necessitating an understanding of how they represent geopolitical identities. In this paper, we analyse personas generated for Palestinian and Israeli identities by five popular LLMs across 640 experimental conditions, varying context (war vs non-war) and assigned roles. We observe significant distributional patterns in the generated attributes: Palestinian profiles in war contexts are frequently associated with lower socioeconomic status and survival-oriented roles, whereas Israeli profiles predominantly retain middle-class status and specialised professional attributes. When prompted with explicit instructions to avoid harmful assumptions, models exhibit diverse distributional changes, e.g., marked increases in non-binary gender inferences or a convergence toward generic occupational roles (e.g., "student"), while the underlying socioeconomic distinctions often remain. Furthermore, analysis of reasoning traces reveals an interesting dynamics between model reasoning and generation: while rationales consistently mention fairness-related concepts, the final generated personas follow the aforementioned diverse distributional changes. These findings illustrate a picture of how models interpret geopolitical contexts, while suggesting that they process fairness and adjust in varied ways; there is no consistent, direct translation of fairness concepts into representative outcomes.