Identity, Cooperation and Framing Effects within Groups of Real and Simulated Humans

TL;DR

This paper explores how large language models can simulate human social behaviors by incorporating identity, context, and framing effects, leading to more accurate replication of human actions in social dilemma scenarios.

Contribution

It demonstrates that deep binding of models with detailed backstories enhances simulation fidelity and enables modeling of contextual factors influencing human behavior.

Findings

Rich narrative conditioning improves simulation accuracy.

Models can replicate effects of framing and temporal context.

Enhanced fidelity aids understanding of human social decision-making.

Abstract

Humans act via a nuanced process that depends both on rational deliberation and also on identity and contextual factors. In this work, we study how large language models (LLMs) can simulate human action in the context of social dilemma games. While prior work has focused on "steering" (weak binding) of chat models to simulate personas, we analyze here how deep binding of base models with extended backstories leads to more faithful replication of identity-based behaviors. Our study has these findings: simulation fidelity vs human studies is improved by conditioning base LMs with rich context of narrative identities and checking consistency using instruction-tuned models. We show that LLMs can also model contextual factors such as time (year that a study was performed), question framing, and participant pool effects. LLMs, therefore, allow us to explore the details that affect human studies but which are often omitted from experiment descriptions, and which hamper accurate replication.