Large Language Models as Psychological Simulators: A Methodological Guide

TL;DR

This paper offers a comprehensive methodological framework for using large language models as psychological simulators, addressing applications, validation, challenges, and ethical considerations in research.

Contribution

It introduces new methods for developing psychologically grounded personas and probing cognitive processes using LLMs, filling a gap in methodological guidance.

Findings

LLMs can simulate diverse psychological roles and personas.

Strategies for validating LLM-based simulations against human data.

Discussion of challenges like prompt sensitivity and ethical issues.

Abstract

Large language models (LLMs) offer emerging opportunities for psychological and behavioral research, but methodological guidance is lacking. This article provides a framework for using LLMs as psychological simulators across two primary applications: simulating roles and personas to explore diverse contexts, and serving as computational models to investigate cognitive processes. For simulation, we present methods for developing psychologically grounded personas that move beyond demographic categories, with strategies for validation against human data and use cases ranging from studying inaccessible populations to prototyping research instruments. For cognitive modeling, we synthesize emerging approaches for probing internal representations, methodological advances in causal interventions, and strategies for relating model behavior to human cognition. We address overarching challenges including prompt sensitivity, temporal limitations from training data cutoffs, and ethical considerations that extend beyond traditional human subjects review. Throughout, we emphasize the need for transparency about model capabilities and constraints. Together, this framework integrates emerging empirical evidence about LLM performance--including systematic biases, cultural limitations, and prompt brittleness--to help researchers wrangle these challenges and leverage the unique capabilities of LLMs in psychological research.