Simulation Experiments as a Causal Problem

TL;DR

This paper proposes viewing simulation experiments as causal interventions on data generating mechanisms, using causal tools and frameworks to improve design, interpretation, and goals in statistical simulations across various domains.

Contribution

It introduces a causal perspective to simulation design, enabling clearer target estimands, improved experiment modifications, and better performance assessment methods.

Findings

Causal tools clarify simulation target estimands.

Graphical models guide simulation modifications.

Causal perspective enhances interpretation of simulation results.

Abstract

Simulation methods are among the most ubiquitous methodological tools in statistical science. In particular, statisticians often is simulation to explore properties of statistical functionals in models for which developed statistical theory is insufficient or to assess finite sample properties of theoretical results. We show that the design of simulation experiments can be viewed from the perspective of causal intervention on a data generating mechanism. We then demonstrate the use of causal tools and frameworks in this context. Our perspective is agnostic to the particular domain of the simulation experiment which increases the potential impact of our proposed approach. In this paper, we consider two illustrative examples. First, we re-examine a predictive machine learning example from a popular textbook designed to assess the relationship between mean function complexity and the mean-squared error. Second, we discuss a traditional causal inference method problem, simulating the effect of unmeasured confounding on estimation, specifically to illustrate bias amplification. In both cases, applying causal principles and using graphical models with parameters and distributions as nodes in the spirit of influence diagrams can 1) make precise which estimand the simulation targets , 2) suggest modifications to better attain the simulation goals, and 3) provide scaffolding to discuss performance criteria for a particular simulation design.