Imputation-based randomization tests for randomized experiments with interference

TL;DR

This paper introduces an imputation-based randomization test for experiments with interference, combining Bayesian ideas with randomization to improve power and control type I error.

Contribution

It proposes a novel imputation-based approach for Fisher randomization tests under interference, with theoretical validity and practical advantages over existing methods.

Findings

Effectively controls type I error rate

Significantly improves testing power

Demonstrated on experiments with interference

Abstract

The presence of interference renders classic Fisher randomization tests infeasible due to nuisance unknowns. To address this issue, we propose imputing the nuisance unknowns and computing Fisher randomization p-values multiple times, then averaging them. We term this approach the imputation-based randomization test and provide theoretical results on its asymptotic validity. Our method leverages the merits of randomization and the flexibility of the Bayesian framework: for multiple imputations, we can either employ the empirical distribution of observed outcomes to achieve robustness against model mis-specification or utilize a parametric model to incorporate prior information. Simulation results demonstrate that our method effectively controls the type I error rate and significantly enhances the testing power compared to existing randomization tests for randomized experiments with interference. We apply our method to a two-round randomized experiment with multiple treatments and one-way interference, where existing randomization tests exhibit limited power.