Can LLMs Emulate Human Belief Dynamics?

TL;DR

This study evaluates whether large language models can replicate human belief formation and change in social networks, revealing significant limitations and conformist biases that differ from human behavior.

Contribution

The paper systematically tests 12 LLMs on belief dynamics, exposing their inability to mimic human initial beliefs and their tendency towards conformity.

Findings

LLMs fail to replicate initial human belief distributions.

LLMs tend to conform more than humans in social settings.

They show nuanced homophilic tendencies within networks.

Abstract

Can LLMs simulate how humans form and change beliefs in social networks? We put this to the test by replicating an established study on belief dynamics, evaluating 12 LLMs across multiple model families and parameter sizes. The answer is a clear no, and in systematic ways. LLMs fail to capture initial human belief distributions and tend to be overall more conformist than humans, shifting their responses to align with those around them. They also take a nuanced approach to emulating human homophilic tendencies within networks. Our findings carry a double payoff: they highlight fundamental properties of LLM behavior, and they raise a sharp warning against deploying LLMs as human proxies in social simulations.