The $\textit{Silicon Society}$ Cookbook: Design Space of LLM-based Social Simulations

TL;DR

This paper systematically analyzes the design choices in LLM-based social simulations, highlighting the complex interactions and the critical impact of the base model on simulation outcomes.

Contribution

It provides a detailed exploration of the design space for Silicon Societies, emphasizing the importance of base model selection and interaction effects.

Findings

The geometry of the design space is non-trivial.

Some parameters interact in complex ways.

The choice of base LLM significantly influences results.

Abstract

Studies attempting to simulate human behavior with $\textit{Silicon Societies}$ grow in numbers while LLM-only social networks have started appearing outside of controlled settings. However, the design space of these networks remains under-studied, which contributes to a gap in validating model realism. To enable future works to make more informed design decisions, we perform a systematic analysis of the consequences and interactions of key design choices in simulated social networks, including the choice of base model used to model individual agents, and how they are connected to each other. Using surveys as a proxy for agent opinions, our findings suggest that the geometry of the design space is non-trivial, with some parameters behaving in additive ways while others display more complex interactions. In particular, the choice of the base LLM is the most important variable impacting the simulation outcomes.