Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity

TL;DR

This study examines how large language models simulate political discourse on social media, revealing that richer context improves consistency but also increases polarization, bias, and toxicity through a phenomenon called generation exaggeration.

Contribution

The paper uncovers how LLMs amplify salient traits and biases in social media simulations, highlighting the impact of contextualization on polarization and toxicity, and revealing structural biases in LLM outputs.

Findings

Richer context improves internal consistency of LLM agents.

Contextualization amplifies polarization, bias, and toxicity.

LLMs reconstruct user traits, reflecting internal biases more than actual behavior.

Abstract

We investigate how Large Language Models (LLMs) behave when simulating political discourse on social media. Leveraging 21 million interactions on X during the 2024 U.S. presidential election, we construct LLM agents based on 1,186 real users, prompting them to reply to politically salient tweets under controlled conditions. Agents are initialized either with minimal ideological cues (Zero Shot) or recent tweet history (Few Shot), allowing one-to-one comparisons with human replies. We evaluate three model families (Gemini, Mistral, and DeepSeek) across linguistic style, ideological consistency, and toxicity. We find that richer contextualization improves internal consistency but also amplifies polarization, stylized signals, and harmful language. We observe an emergent distortion that we call "generation exaggeration": a systematic amplification of salient traits beyond empirical baselines. Our analysis shows that LLMs do not emulate users, they reconstruct them. Their outputs, indeed, reflect internal optimization dynamics more than observed behavior, introducing structural biases that compromise their reliability as social proxies. This challenges their use in content moderation, deliberative simulations, and policy modeling.