Optimized variance estimation under interference and complex experimental designs

TL;DR

This paper introduces a convex optimization approach to construct minimally conservative variance estimators for complex experimental designs with interference, improving inference accuracy.

Contribution

It formulates the variance estimation problem as a convex optimization task to find the lowest conservative bound, enhancing existing methods.

Findings

Estimators are less conservative with accurate background knowledge.

Numerical results show significant reduction in conservativeness.

Method guarantees conservative estimates regardless of background knowledge accuracy.

Abstract

Unbiased and consistent variance estimators generally do not exist for design-based treatment effect estimators because experimenters never observe more than one potential outcome for any unit. The problem is exacerbated by interference and complex experimental designs. Experimenters must accept conservative variance estimators in these settings, but they can strive to minimize conservativeness. In this paper, we show that the task of constructing a minimally conservative variance estimator can be interpreted as an optimization problem that aims to find the lowest estimable upper bound of the true variance given the experimenter's risk preference and knowledge of the potential outcomes. We characterize the set of admissible bounds in the class of quadratic forms, and we demonstrate that the optimization problem is a convex program for many natural objectives. The resulting variance estimators are guaranteed to be conservative regardless of whether the background knowledge used to construct the bound is correct, but the estimators are less conservative if the provided information is reasonably accurate. Numerical results show that the resulting variance estimators can be considerably less conservative than existing estimators, allowing experimenters to draw more informative inferences about treatment effects.