Can We Validate Counterfactual Estimations in the Presence of General Network Interference?

TL;DR

This paper introduces a novel framework with a network bootstrap and counterfactual cross-validation to validate causal estimations in experiments with network interference, supported by a comprehensive benchmark suite.

Contribution

It presents a theoretically grounded network bootstrap and a new validation procedure for causal inference under interference, extending message-passing models with heterogeneous data.

Findings

The framework enables reliable validation of causal estimators in network interference settings.

The network bootstrap generates multiple valid subpopulations from a single experiment.

Extensive experiments demonstrate robustness of the method across diverse network scenarios.

Abstract

Randomized experiments have become a cornerstone of evidence-based decision-making in contexts ranging from online platforms to public health. However, in experimental settings with network interference, a unit's treatment can influence outcomes of other units, challenging both causal effect estimation and its validation. Classic validation approaches fail as outcomes are only observable under a single treatment scenario and exhibit complex correlation patterns due to interference. To address these challenges, we introduce a framework that facilitates the use of machine learning tools for both estimation and validation in causal inference. Central to our approach is the new distribution-preserving network bootstrap, a theoretically-grounded technique that generates multiple statistically-valid subpopulations from a single experiment's data. This amplification of experimental samples enables our second contribution: a counterfactual cross-validation procedure. This procedure adapts the principles of model validation to the unique constraints of causal settings, providing a rigorous, data-driven method for selecting and evaluating estimators. We extend recent causal message-passing developments by incorporating heterogeneous unit-level characteristics and varying local interactions, ensuring reliable finite-sample performance through non-asymptotic analysis. Additionally, we develop and publicly release a comprehensive benchmark toolbox featuring diverse experimental environments, from networks of interacting AI agents to ride-sharing applications. These environments provide known ground truth values while maintaining realistic complexities, enabling systematic evaluation of causal inference methods. Extensive testing across these environments demonstrates our method's robustness to diverse forms of network interference.