From Script to Stage: Automating Experimental Design for Social Simulations with LLMs

TL;DR

This paper introduces FSTS, an automated framework leveraging LLMs to simplify and enhance the design of social science experiments through script generation and multi-agent simulation.

Contribution

FSTS automates experimental design in social simulations, reducing reliance on expert knowledge and improving scientific rigor using a structured script-based approach.

Findings

Agents can enact scripts accurately within the simulated environment.

The framework reproduces results consistent with real-world social phenomena.

FSTS lowers barriers for social science experimental design.

Abstract

Multi-agent simulation based on LLMs has increasingly emerged as a new paradigm for exploring complex social phenomena and validating theoretical hypotheses. However, traditional experimental design in the social sciences relies heavily on interdisciplinary expert knowledge, involving cumbersome procedures and high technical barriers. While LLM-driven agents demonstrate broad prospects for designing experiments, their limitations regarding reliability and scientific rigor continue to significantly hinder their in-depth application in social science research. To address these challenges, this paper proposes FSTS, an automated framework for multi-agent experiment design based on script generation. Drawing on the concept of the "Decision Theater," the framework deconstructs experimental design into three core phases: Script Composition, Script Finalization, and Actor Generation. Tests across multiple scenarios indicate that the agents generated by this framework can enact the script within the "experimental theater", reproducing results consistent with real-world situations. The proposal of FSTS not only effectively lowers the barrier for social science experimental design but also provides scientifically grounded decision support for policy-making.