Diagnosing Korean-Language LLM Political Bias via Census-Grounded Agent Simulation

TL;DR

This paper introduces Dynamo-K, a framework for diagnosing political biases in Korean-language LLMs through simulations, revealing systematic failure modes and proposing calibration methods, with validation on election predictions.

Contribution

The paper presents Dynamo-K, a novel, open-source simulation framework for diagnosing and addressing political biases in Korean-language LLMs, including new calibration techniques.

Findings

Mitigation reduces MAE by 5.2 times in moderate agents.

Scenario reframing recovers 62% of 2017 MAE.

Reweighting adapter calibrates models without candidate names.

Abstract

Large language models (LLMs) exhibit systematic political biases in voter simulations, but their underlying mechanisms and cross-lingual generalizations remain poorly understood. We introduce Dynamo-K, a census-grounded simulation framework evaluating Korean-language LLM political behavior across four models on six Korean elections (2017-2025). Using this framework, we identify three systematic failure modes: (1) progressive bias in moderate agents, where explicit mitigation reduces Mean Absolute Error (MAE) by 5.2 times; (2) model-dependent third-party salience collapse, distinguishing between salience failure and decision bias; and (3) regional polarization collapse, where models bidirectionally under-predict historical party strongholds. To address these failures, we demonstrate that scenario reframing recovers 62% of 2017 MAE by restoring third-party visibility. Furthermore, we introduce a learned reweighting adapter that successfully calibrates opposing-valence models without relying on candidate names at train or test time. Validating our diagnostic framework, Dynamo-K accurately predicts 3/3 presidential winners - including a 2.1%p MAE on the highly contested 0.73%p-margin 2022 race - and correctly identifies the dominant party in a held-out local election. The pipeline is open-source and provides a scalable, cost-effective method for diagnosing LLM political behavior.