Design-based Estimation Theory for Complex Experiments

TL;DR

This paper develops a comprehensive design-based estimation framework for complex randomized experiments, including those with interference, providing new estimators, variance estimation procedures, and a measure of experimental complexity.

Contribution

It introduces a general theory for treatment effect estimation in complex designs, with new estimators and variance bounds, applicable to practical experiments with interference.

Findings

New estimators with favorable asymptotic properties

Procedures for consistent variance estimation

A scalar measure of experimental complexity

Abstract

This paper considers the estimation of treatment effects in randomized experiments with complex experimental designs, including cases with interference between units. We develop a design-based estimation theory for arbitrary experimental designs. Our theory facilitates the analysis of many design-estimator pairs that researchers commonly employ in practice and provide procedures to consistently estimate asymptotic variance bounds. We propose new classes of estimators with favorable asymptotic properties from a design-based point of view. In addition, we propose a scalar measure of experimental complexity which can be linked to the design-based variance of the estimators. We demonstrate the performance of our estimators using simulated datasets based on an actual network experiment studying the effect of social networks on insurance adoptions.