Using Large Language Models to Simulate Human Behavioural Experiments: Port of Mars

TL;DR

This paper explores using large language models to simulate human behavior in collective risk social dilemmas, specifically in the Port of Mars experiment, to enable scalable and diverse empirical research.

Contribution

It demonstrates the feasibility and validity of employing LLMs to replicate complex social dilemma experiments like Port of Mars, expanding research capabilities.

Findings

LLMs can simulate diverse human-like responses in CRSD scenarios.

The approach offers a scalable alternative to large-scale human experiments.

Preliminary results show comparable patterns to human data in Port of Mars.

Abstract

Collective risk social dilemmas (CRSD) highlight a trade-off between individual preferences and the need for all to contribute toward achieving a group objective. Problems such as climate change are in this category, and so it is critical to understand their social underpinnings. However, rigorous CRSD methodology often demands large-scale human experiments but it is difficult to guarantee sufficient power and heterogeneity over socio-demographic factors. Generative AI offers a potential complementary approach to address thisproblem. By replacing human participants with large language models (LLM), it allows for a scalable empirical framework. This paper focuses on the validity of this approach and whether it is feasible to represent a large-scale human-like experiment with sufficient diversity using LLM. In particular, where previous literature has focused on political surveys, virtual towns and classical game-theoretic examples, we focus on a complex CRSD used in the institutional economics and sustainability literature known as Port of Mars