Reasoning Boosts Opinion Alignment in LLMs

TL;DR

This paper investigates how structured reasoning can improve large language models' ability to produce opinion-aligned responses in political contexts, demonstrating that reasoning enhances bias mitigation but does not eliminate it entirely.

Contribution

The study introduces a reasoning-based training approach for LLMs to improve opinion alignment and provides datasets and benchmarks for future research.

Findings

Reasoning improves opinion alignment in LLMs.

Models trained with reasoning are competitive with strong baselines.

Bias is reduced but not fully eliminated by reasoning.

Abstract

Opinion modeling aims to capture individual or group political preferences, enabling applications such as digital democracies, where models could help shape fairer and more popular policies. Given their versatility, strong generalization capabilities, and demonstrated success across diverse text-to-text applications, large language models (LLMs) are natural candidates for this task. However, due to their statistical nature and limited causal understanding, they tend to produce biased opinions when prompted naively. In this work, we study whether reasoning can improve opinion alignment. Motivated by the recent advancement in mathematical reasoning enabled by reinforcement learning (RL), we train models to produce profile-consistent answers through structured reasoning. We evaluate our approach on three datasets covering U.S., European, and Swiss politics. Results indicate that reasoning enhances opinion modeling and is competitive with strong baselines, but does not fully remove bias, highlighting the need for additional mechanisms to build faithful political digital twins using LLMs. By releasing both our method and datasets, we establish a solid baseline to support future research on LLM opinion alignment.