Digital Homunculi and Institutional Design: Breaking Through the Experimentation Bottleneck

TL;DR

This paper proposes using GenAI-powered agents, called digital homunculi, to simulate human behavior in institutional settings, potentially overcoming the slow and costly traditional experimentation process in democracy research.

Contribution

It introduces the digital homunculi methodology, leveraging large language models to generate synthetic data for testing institutional innovations rapidly and ethically.

Findings

Digital homunculi can elicit human-like behavior in multi-agent simulations.

Proposes validation strategies including behavioral back-testing.

Outlines infrastructure needs for rigorous evaluation.

Abstract

Democracy research faces a longstanding experimentation bottleneck. Potential institutional innovations remain untested because human-subject studies are slow, expensive, and ethically fraught. This paper argues that digital homunculi, that is, GenAI-powered agents role-playing humans in diverse institutional settings, could offer a way to break through the bottleneck. In contrast to the legacy agent-based modeling, building complexity from transparent simple rules, the digital homunculi methodology aims to extract latent human behavioral knowledge from opaque large language models. To this ends, it designs multi-agent interactions as elicitation devices to trigger in LLMs human-like behavior that can be recorded as synthetic data. However, the validity of synthetic data remains an open question. Success requires that accurate, coherent, transferable models of humans ('little humans' - homunculi) already lurk within GenAI's inscrutable matrices and can be lured out via the social simulation role-play exercise. At the same time, to the extent these attempts are successful, they promise to completely transform the political economy of institutional research from scarcity to abundance. To help mitigate the number of challenges along the way to such success, I propose concrete validation strategies including behavioral back-testing via knowledge cutoffs, and outline infrastructure requirements for rigorous evaluation. The stakes are high: legacy democratic institutions develop at much slower pace than the surrounding technological landscape. If they falter, we lack a repository of tested backup alternatives. Breaking through the experimentation bottleneck must be a priority and digital homunculi may be quickly maturing into a methodology capable of achieving this feat.