Noise-free comparison of stochastic agent-based simulations using common random numbers

TL;DR

This paper introduces a novel method to eliminate stochastic noise caused by misaligned random numbers in agent-based models, enabling more reliable comparison of simulation scenarios and reducing the number of simulations needed.

Contribution

The paper presents a new methodology that removes noise from random number misalignment in agent-based models, improving analysis accuracy and efficiency.

Findings

Reduces the number of simulations needed by over 10-fold in some cases

Enables meaningful individual-level comparison between scenarios

Demonstrated across three diverse examples

Abstract

Random numbers are at the heart of every agent-based model (ABM) of health and disease. By representing each individual in a synthetic population, agent-based models enable detailed analysis of intervention impact and parameter sensitivity. Yet agent-based modeling has a fundamental signal-to-noise problem, in which small changes between simulations cannot be reliably differentiated from stochastic noise resulting from misaligned random number realizations. We introduce a novel methodology that eliminates noise due to misaligned random numbers, a first for agent-based modeling. Our approach enables meaningful individual-level analysis between ABM scenarios because all differences are driven by mechanistic effects rather than random number noise. We demonstrate the benefits of our approach on three disparate examples. Results consistently show reductions in the number of simulations required to achieve a given standard error with levels exceeding 10-fold for some applications.