Emulating Aggregate Human Choice Behavior and Biases with GPT Conversational Agents

TL;DR

This study investigates whether GPT-based conversational agents can accurately emulate individual human decision biases and their dynamics under varying contextual factors like cognitive load.

Contribution

The paper demonstrates that GPT-4 and GPT-5 can replicate human decision biases in interactive scenarios, revealing differences between models and implications for bias-aware AI systems.

Findings

LLMs reproduce human biases with high accuracy

Differences observed between GPT-4 and GPT-5 in emulating biases

Robust biases identified under different dialogue complexities

Abstract

Cognitive biases often shape human decisions. While large language models (LLMs) have been shown to reproduce well-known biases, a more critical question is whether LLMs can predict biases at the individual level and emulate the dynamics of biased human behavior when contextual factors, such as cognitive load, interact with these biases. We adapted three well-established decision scenarios into a conversational setting and conducted a human experiment (N=1100). Participants engaged with a chatbot that facilitates decision-making through simple or complex dialogues. Results revealed robust biases. To evaluate how LLMs emulate human decision-making under similar interactive conditions, we used participant demographics and dialogue transcripts to simulate these conditions with LLMs based on GPT-4 and GPT-5. The LLMs reproduced human biases with precision. We found notable differences between models in how they aligned human behavior. This has important implications for designing and evaluating adaptive, bias-aware LLM-based AI systems in interactive contexts.