Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias

TL;DR

This study systematically audits 26 large language models to analyze their political biases and psychometric profiles across multiple dimensions, revealing mostly libertarian-left tendencies and emphasizing the importance of multidimensional evaluation frameworks.

Contribution

It introduces a comprehensive, multidimensional auditing framework for assessing political alignment in large language models, highlighting the limitations of single-axis evaluations.

Findings

Most models cluster in the Libertarian-Left quadrant of the Political Compass.

Model identity explains most variance across prompt variants ($ta^2 > 0.90$).

Psychometric ideological positioning does not predict classification errors.

Abstract

As large language models (LLMs) are increasingly deployed, understanding how they express political positioning is important for evaluating alignment and downstream effects. We audit 26 contemporary LLMs using three political psychometric inventories (Political Compass, SapplyValues, 8Values) and a news bias labeling task. To test robustness, inventories are administered across multiple semantic prompt variants and analyzed with a two-way ANOVA separating model and prompt effects. Most models cluster in a similar ideological region, with 96.3% located in the Libertarian-Left quadrant of the Political Compass, and model identity explaining most variance across prompt variants ($\eta^2 > 0.90$). Cross-instrument comparisons suggest that the Political Compass social axis aligns more strongly with cultural progressivism than authority-related measures ($r=-0.64$). We observe differences between open-weight and closed-source models and asymmetric performance in detecting extreme political bias in downstream classification. Regression analysis finds that psychometric ideological positioning does not significantly predict classification errors, providing no evidence of a statistically significant relationship between conversational ideological identity and task-level behavior. These findings suggest that single-axis evaluations are insufficient and that multidimensional auditing frameworks are important to characterize alignment behavior in deployed LLMs. Our code and data are publicly available at https://github.com/sakhadib/PolAlignLLM.