Network Effects and Agreement Drift in LLM Debates

TL;DR

This paper investigates how LLM agents behave in simulated debates, revealing a phenomenon called agreement drift where agents tend to shift toward certain opinions, emphasizing the importance of understanding structural effects and biases.

Contribution

It introduces a network generation model to study LLM debate dynamics and identifies the agreement drift phenomenon affecting opinion shifts.

Findings

Agents exhibit a directional agreement drift in debates.

Structural effects influence LLM behavior more than biases.

Highlighting the need to disentangle structural effects from biases.

Abstract

Large Language Models (LLMs) have demonstrated an unprecedented ability to simulate human-like social behaviors, making them useful tools for simulating complex social systems. However, it remains unclear to what extent these simulations can be trusted to accurately capture key social mechanisms, particularly in highly unbalanced contexts involving minority groups. This paper uses a network generation model with controlled homophily and class sizes to examine how LLM agents behave collectively in multi-round debates. Moreover, our findings highlight a particular directional susceptibility that we term \textit{agreement drift}, in which agents are more likely to shift toward specific positions on the opinion scale. Overall, our findings highlight the need to disentangle structural effects from model biases before treating LLM populations as behavioral proxies for human groups.