Validating Generative Agent-Based Models of Social Norm Enforcement: From Replication to Novel Predictions

TL;DR

This paper develops a systematic validation framework for generative agent-based models of social norm enforcement, demonstrating their ability to replicate known behaviors and generate novel, testable social science predictions.

Contribution

It introduces a two-stage validation approach for LLM-based social behavior models and shows how validated models can produce new insights into social norm enforcement.

Findings

Both persona-based differences and theory of mind are essential for replicating third-party punishment.

Reputational information via gossip increases cooperation in public goods games.

Anonymous punishment reduces third-party punishment rates but does not eliminate them.

Abstract

As large language models (LLMs) advance, there is growing interest in using them to simulate human social behavior through generative agent-based modeling (GABM). However, validating these models remains a key challenge. We present a systematic two-stage validation approach using social dilemma paradigms from psychological literature, first identifying the cognitive components necessary for LLM agents to reproduce known human behaviors in mixed-motive settings from two landmark papers, then using the validated architecture to simulate novel conditions. Our model comparison of different cognitive architectures shows that both persona-based individual differences and theory of mind capabilities are essential for replicating third-party punishment (TPP) as a costly signal of trustworthiness. For the second study on public goods games, this architecture is able to replicate an increase in cooperation from the spread of reputational information through gossip. However, an additional strategic component is necessary to replicate the additional boost in cooperation rates in the condition that allows both ostracism and gossip. We then test novel predictions for each paper with our validated generative agents. We find that TPP rates significantly drop in settings where punishment is anonymous, yet a substantial amount of TPP persists, suggesting that both reputational and intrinsic moral motivations play a role in this behavior. For the second paper, we introduce a novel intervention and see that open discussion periods before rounds of the public goods game further increase contributions, allowing groups to develop social norms for cooperation. This work provides a framework for validating generative agent models while demonstrating their potential to generate novel and testable insights into human social behavior.