LLMs as Strategic Actors: Behavioral Alignment, Risk Calibration, and Argumentation Framing in Geopolitical Simulations

TL;DR

This study evaluates how large language models behave as strategic agents in geopolitical simulations, comparing their decision-making, risk assessment, and argumentation to humans, revealing similarities and key differences in strategic profiles.

Contribution

It provides a comprehensive analysis of LLMs' strategic behavior in geopolitical contexts, highlighting their normative framing and divergence from human strategies over time.

Findings

Models approximate human decisions in initial rounds

Models show normative-cooperative framing in explanations

Behavior diverges from humans as simulations progress

Abstract

Large language models (LLMs) are increasingly proposed as agents in strategic decision environments, yet their behavior in structured geopolitical simulations remains under-researched. We evaluate six popular state-of-the-art LLMs alongside results from human results across four real-world crisis simulation scenarios, requiring models to select predefined actions and justify their decisions across multiple rounds. We compare models to humans in action alignment, risk calibration through chosen actions' severity, and argumentative framing grounded in international relations theory. Results show that models approximate human decision patterns in base simulation rounds but diverge over time, displaying distinct behavioural profiles and strategy updates. LLM explanations for chosen actions across all models exhibit a strong normative-cooperative framing centered on stability, coordination, and risk mitigation, with limited adversarial reasoning.