Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models

TL;DR

This paper demonstrates that zero-shot persona prompting with large language models can effectively simulate and predict voting behavior of European Parliament members, providing a new tool for political analysis.

Contribution

It introduces a novel approach using persona prompts to accurately predict European Parliament voting behavior with LLMs, validated through extensive evaluation.

Findings

Achieved a weighted F1 score of approximately 0.793 in voting prediction.

Predictions remain stable under counterfactual and different prompting conditions.

Provided a new dataset of European Parliament politicians for research.

Abstract

Large Language Models (LLMs) display remarkable capabilities to understand or even produce political discourse but have been found to consistently exhibit a progressive left-leaning bias. At the same time, so-called persona or identity prompts have been shown to produce LLM behavior that aligns with socioeconomic groups with which the base model is not aligned. In this work, we analyze whether zero-shot persona prompting with limited information can accurately predict individual voting decisions and, by aggregation, accurately predict the positions of European groups on a diverse set of policies. We evaluate whether predictions are stable in response to counterfactual arguments, different persona prompts, and generation methods. Finally, we find that we can simulate the voting behavior of Members of the European Parliament reasonably well, achieving a weighted F1 score of approximately 0.793. Our persona dataset of politicians in the 2024 European Parliament and our code are available at the following url: https://github.com/dess-mannheim/european_parliament_simulation.