SLALOM: Simulation Lifecycle Analysis via Longitudinal Observation Metrics for Social Simulation

TL;DR

SLALOM introduces a new framework for validating social simulations by analyzing their process fidelity through longitudinal metrics, improving the assessment of social mechanism plausibility.

Contribution

It proposes a novel validation approach that emphasizes process fidelity and uses Dynamic Time Warping to compare simulated and real social trajectories.

Findings

SLALOM provides a quantitative measure of structural realism in social simulations.

The framework helps distinguish plausible social dynamics from stochastic noise.

It offers a pathway to more robust policy simulation standards.

Abstract

Large Language Model (LLM) agents offer a potentially-transformative path forward for generative social science but face a critical crisis of validity. Current simulation evaluation methodologies suffer from the "stopped clock" problem: they confirm that a simulation reached the correct final outcome while ignoring whether the trajectory leading to it was sociologically plausible. Because the internal reasoning of LLMs is opaque, verifying the "black box" of social mechanisms remains a persistent challenge. In this paper, we introduce SLALOM (Simulation Lifecycle Analysis via Longitudinal Observation Metrics), a framework that shifts validation from outcome verification to process fidelity. Drawing on Pattern-Oriented Modeling (POM), SLALOM treats social phenomena as multivariate time series that must traverse specific SLALOM gates, or intermediate waypoint constraints representing distinct phases. By utilizing Dynamic Time Warping (DTW) to align simulated trajectories with empirical ground truth, SLALOM offers a quantitative metric to assess structural realism, helping to differentiate plausible social dynamics from stochastic noise and contributing to more robust policy simulation standards.