On the Impossibility of Specification Testing of Interference Models Based on Exposure Mappings

TL;DR

This paper proves that no statistical test can reliably distinguish correct from incorrect interference models based on exposure mappings, highlighting fundamental limitations in causal spillover analysis.

Contribution

It establishes a strong impossibility result for specification testing of interference models, showing the sum of Type I and II errors is always one under broad conditions.

Findings

Any test aiming for power against larger models has worst-case errors summing to one.

Existing tests controlling Type I error have limited power to detect misspecification.

Provides a uniformly consistent test for no-interference versus network-linear-in-means models.

Abstract

In order to estimate causal effects in a randomized experiment where spillovers are suspected to occur, analysts must posit a model of interference. The most popular class of interference models are those based on exposure mappings. In practice, it is rarely clear which interference model accurately captures the true nature of spillovers in the experiment. In response, researchers have developed specification tests which seek to determine whether a given interference model is correctly specified. In this context, Type I error is the rejection rate when the interference model is actually correct and Type II error is the acceptance rate when the interference model is incorrectly specified. While existing tests have been explicitly constructed to control Type I error, their Type II error remains less well understood. In this paper, we provide a strong impossibility result: any specification test for an exposure mapping model which aims to have power against a larger exposure mapping model has worst-case Type I and Type II errors that sum to one. This means that no specification test can provide uniformly better performance than the naive test which discards all data and rejects the null at random. Our negative result holds for all sample sizes, for uniformly bounded outcomes, and for alternatives which are maximally separated from the null. Informative specification tests must therefore further restrict the alternative model against which they seek to attain power. To this end, we provide a uniformly consistent test for differentiating no-interference from a network-linear-in-means model.