Estimating the prevalance of indirect effects and other spillovers

TL;DR

This paper introduces methods to estimate how widespread indirect effects are in experiments with potential interference, providing conservative estimates without relying on strong assumptions.

Contribution

It develops conservative estimation techniques for the prevalence of indirect effects using randomization, applicable even when interference models are uncertain.

Findings

Significant fraction of students affected by treatments in other schools.

Methods work under minimal assumptions, only requiring randomization.

Provides a central limit theorem for asymptotic coverage.

Abstract

In settings where interference between units is possible, we define the prevalence of indirect effects to be the number of units who are affected by the treatment of others. This quantity does not fully identify an indirect effect, but may be used to show whether such effects are widely prevalent. Given a randomized experiment with binary-valued outcomes, methods are presented for conservative point estimation and one-sided interval estimation. No assumptions beyond randomization of treatment are required, allowing for usage in settings where models or assumptions on interference might be questionable. To show asymptotic coverage of our intervals in settings not covered by existing results, we provide a central limit theorem that combines local dependence and sampling without replacement. Consistency and minimax properties of the point estimator are shown as well. The approach is demonstrated on an experiment in which students were treated for a highly transmissible parasitic infection, for which we find that a significant fraction of students were affected by the treatment of schools other than their own.