Validating Causal Message Passing Against Network-Aware Methods on Real Experiments

TL;DR

This paper shows that causal message passing, which uses outcome data over time rather than network structure, can accurately estimate treatment effects in network experiments even without network data.

Contribution

It demonstrates that causal message passing can effectively replace network-aware methods for estimating spillover effects using only temporal outcome data.

Findings

Causal message passing matches network-aware methods in effect direction.

It produces statistically significant estimates comparable to network-based approaches.

The method is effective even with incomplete or unreliable network data.

Abstract

Estimating total treatment effects in the presence of network interference typically requires knowledge of the underlying interaction structure. However, in many practical settings, network data is either unavailable, incomplete, or measured with substantial error. We demonstrate that causal message passing, a methodology that leverages temporal structure in outcome data rather than network topology, can recover total treatment effects comparable to network-aware approaches. We apply causal message passing to two large-scale field experiments where a recently developed bipartite graph methodology, which requires network knowledge, serves as a benchmark. Despite having no access to the interaction network, causal message passing produces effect estimates that match the network-aware approach in direction across all metrics and in statistical significance for the primary decision metric. Our findings validate the premise of causal message passing: that temporal variation in outcomes can serve as an effective substitute for network observation when estimating spillover effects. This has important practical implications: practitioners facing settings where network data is costly to collect, proprietary, or unreliable can instead exploit the temporal dynamics of their experimental data.