When Does Pairing Seeds Reduce Variance? Evidence from a Multi-Agent Economic Simulation

TL;DR

This paper investigates how using shared random seeds in multi-agent economic simulations can reduce variance in comparative evaluations, revealing systematic differences that are often hidden under traditional independent testing.

Contribution

It provides a theoretical and empirical analysis of seed pairing effects, demonstrating variance reduction and systematic outcome differences in multi-agent simulations.

Findings

Shared seeds induce matched stochastic outcomes, reducing variance.

Paired evaluation reveals differences not seen in independent runs.

Variance reduction depends on outcome correlation at the seed level.

Abstract

Machine learning systems appear stochastic but are deterministically random, as seeded pseudorandom number generators produce identical realisations across repeated executions. Standard evaluation practice typically treats runs across alternatives as independent and does not exploit shared sources of randomness. This paper analyses the statistical structure of comparative evaluation under shared random seeds. Under this design, competing systems are evaluated using identical seeds, inducing matched stochastic realisations and yielding strict variance reduction whenever outcomes are positively correlated at the seed level. We demonstrate these effects using an extended learning-based multi-agent economic simulator, where paired evaluation exposes systematic differences in aggregate and distributional outcomes that remain statistically inconclusive under independent evaluation at fixed budgets.