Revealing Political Bias in LLMs through Structured Multi-Agent Debate

TL;DR

This study explores how different large language models and agent gender attributes influence political bias and interaction dynamics in structured multi-agent debates on sensitive topics.

Contribution

It introduces a systematic framework to analyze political bias in LLMs, revealing how model type and gender affect debate attitudes and echo chamber formation.

Findings

Neutral agents tend to align with Democrats.

Gender influences agent opinions and attitude shifts.

Shared political affiliations can lead to echo chambers.

Abstract

Large language models (LLMs) are increasingly used to simulate social behaviour, yet their political biases and interaction dynamics in debates remain underexplored. We investigate how LLM type and agent gender attributes influence political bias using a structured multi-agent debate framework, by engaging Neutral, Republican, and Democrat American LLM agents in debates on politically sensitive topics. We systematically vary the underlying LLMs, agent genders, and debate formats to examine how model provenance and agent personas influence political bias and attitudes throughout debates. We find that Neutral agents consistently align with Democrats, while Republicans shift closer to the Neutral; gender influences agent attitudes, with agents adapting their opinions when aware of other agents' genders; and contrary to prior research, agents with shared political affiliations can form echo chambers, exhibiting the expected intensification of attitudes as debates progress.