# Limitations of design-based causal inference and A/B testing under arbitrary and network interference

## TL;DR

This paper demonstrates fundamental limitations in design-based causal inference and A/B testing when interference between units is arbitrary or network-based, showing that large samples do not guarantee low variance or unbiased estimates.

## Contribution

It formalizes the limitations of nonparametric and parametric inference under interference, revealing that assumptions are necessary for reliable causal estimation in network experiments.

## Key findings

- Unbiased estimators cannot guarantee decreasing variance under interference.
- Large sample sizes do not ensure small variance in experiments with interference.
- Traditional causal inference assumptions do not extend straightforwardly to network interference settings.

## Abstract

Randomized experiments on a network often involve interference between connected units; i.e., a situation in which an individual's treatment can affect the response of another individual. Current approaches to deal with interference, in theory and in practice, often make restrictive assumptions on its structure---for instance, assuming that interference is local---even when using otherwise nonparametric inference strategies. This reliance on explicit restrictions on the interference mechanism suggests a shared intuition that inference is impossible without any assumptions on the interference structure. In this paper, we begin by formalizing this intuition in the context of a classical nonparametric approach to inference, referred to as design-based inference of causal effects. Next, we show how, always in the context of design-based inference, even parametric structural assumptions that allow the existence of unbiased estimators, cannot guarantee a decreasing variance even in the large sample limit. This lack of concentration in large samples is often observed empirically, in randomized experiments in which interference of some form is expected to be present. This result has direct consequences for the design and analysis of large experiments---for instance, in online social platforms---where the belief is that large sample sizes automatically guarantee small variance. More broadly, our results suggest that although strategies for causal inference in the presence of interference borrow their formalism and main concepts from the traditional causal inference literature, much of the intuition from the no-interference case do not easily transfer to the interference setting.