A General Framework for Design-Based Treatment Effect Estimation in Paired Cluster-Randomized Experiments

TL;DR

This paper introduces a comprehensive framework for estimating treatment effects in paired cluster-randomized experiments, addressing analysis challenges and comparing strategies through simulations and real data.

Contribution

It provides a novel, intuitive framework for design-based estimation in pCRTs, emphasizing covariate adjustment and offering unbiased, conservative variance estimators.

Findings

Covariate adjustment improves estimation precision.

Proposed estimators are unbiased and conservatively estimate variance.

Simulation studies demonstrate estimator performance in practice.

Abstract

Paired cluster-randomized experiments (pCRTs) are common across many disciplines because there is often natural clustering of individuals, and paired randomization can help balance baseline covariates to improve experimental precision. Although pCRTs are common, there is surprisingly no obvious way to analyze this randomization design if an individual-level (rather than cluster-level) treatment effect is of interest. Variance estimation is also complicated due to the dependency created through pairing clusters. Therefore, we aim to provide an intuitive and practical comparison between different estimation strategies in pCRTs in order to inform practitioners' choice of strategy. To this end, we present a general framework for design-based estimation in pCRTs for average individual effects. This framework offers a novel and intuitive view on the bias-variance trade-off between estimators and emphasizes the benefits of covariate adjustment for estimation with pCRTs. In addition to providing a general framework for estimation in pCRTs, the point and variance estimators we present support fixed-sample unbiased estimation with similar precision to a common regression model and consistently conservative variance estimation. Through simulation studies, we compare the performance of the point and variance estimators reviewed. Finally, we compare the performance of estimators with simulations using real data from an educational efficacy trial. Our analysis and simulation studies inform the choice of point and variance estimators for analyzing pCRTs in practice.