A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science

TL;DR

This paper introduces a large-scale, theory-driven simulation framework using LLMs to model voter decision-making in political science, addressing data limitations and improving accuracy in election predictions.

Contribution

It presents a novel multi-step reasoning framework integrating demographic, temporal, and ideological factors for large-scale political simulations using LLMs.

Findings

Enhanced simulation accuracy compared to previous models

Robustness across different LLMs demonstrated

Insights into limitations of LLMs in political modeling

Abstract

While LLMs have demonstrated remarkable capabilities in text generation and reasoning, their ability to simulate human decision-making -- particularly in political contexts -- remains an open question. However, modeling voter behavior presents unique challenges due to limited voter-level data, evolving political landscapes, and the complexity of human reasoning. In this study, we develop a theory-driven, multi-step reasoning framework that integrates demographic, temporal and ideological factors to simulate voter decision-making at scale. Using synthetic personas calibrated to real-world voter data, we conduct large-scale simulations of recent U.S. presidential elections. Our method significantly improves simulation accuracy while mitigating model biases. We examine its robustness by comparing performance across different LLMs. We further investigate the challenges and constraints that arise from LLM-based political simulations. Our work provides both a scalable framework for modeling political decision-making behavior and insights into the promise and limitations of using LLMs in political science research.