General Social Agents

TL;DR

This paper introduces a method for creating general AI social agents that predict human behavior in new settings using theory-grounded instructions and minimal data, outperforming traditional models.

Contribution

It presents a novel approach for building versatile social agents that leverage theory, existing data, and AI training to predict human behavior across diverse settings.

Findings

AI agents outperform cognitive hierarchy models in predicting human play.

Agents predict responses in novel games better than existing game-theoretic models.

Simulations match human responses more accurately than prior data-based methods.

Abstract

Useful social science theories predict behavior across settings. However, applying a theory to make predictions in new settings is challenging: rarely can it be done without ad hoc modifications to account for setting-specific factors. We argue that AI agents put in simulations of those novel settings offer an alternative for applying theory, requiring minimal or no modifications. We present an approach for building such "general" agents that use theory-grounded natural language instructions, existing empirical data, and knowledge acquired by the underlying AI during training. To demonstrate the approach in settings where no data from that data-generating process exists--as is often the case in applied prediction problems--we design a heterogeneous population of 883,320 novel games. AI agents are constructed using human data from a small set of conceptually related but structurally distinct "seed" games. In preregistered experiments, on average, agents predict initial human play in a random sample of 1,500 games from the population better than (i) a cognitive hierarchy model, (ii) game-theoretic equilibria, and (iii) out-of-the-box agents. For a small set of separate novel games, these simulations predict responses from a new sample of human subjects better even than the most plausibly relevant published human data.