Social Simulations with Large Language Model Risk Utopian Illusion
Ning Bian, Xianpei Han, Hongyu Lin, Baolei Wu, Jun Wang

TL;DR
This paper systematically analyzes how large language models simulate social behavior, revealing they produce overly idealized, biased representations that lack real human complexity, thus risking misinterpretation in social science applications.
Contribution
It introduces a novel framework for analyzing LLMs' social simulation behavior and uncovers their tendency to generate utopian, biased social interactions unlike genuine human behavior.
Findings
LLMs reflect social desirability bias in simulations
LLMs exhibit social role bias, primacy effect, positivity bias
Simulated societies lack real human complexity and variability
Abstract
Reliable simulation of human behavior is essential for explaining, predicting, and intervening in our society. Recent advances in large language models (LLMs) have shown promise in emulating human behaviors, interactions, and decision-making, offering a powerful new lens for social science studies. However, the extent to which LLMs diverge from authentic human behavior in social contexts remains underexplored, posing risks of misinterpretation in scientific studies and unintended consequences in real-world applications. Here, we introduce a systematic framework for analyzing LLMs' behavior in social simulation. Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions, providing a simple yet effective method to examine emergent social cognitive biases. We conduct extensive experiments involving eight…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLanguage and cultural evolution · Computational and Text Analysis Methods · Topic Modeling
