Bridging Stratification and Regression Adjustment: Batch-Adaptive Stratification with Post-Design Adjustment in Randomized Experiments

TL;DR

This paper introduces an adaptive stratification method for randomized experiments that leverages covariate-outcome relationships to improve statistical efficiency, complementing regression adjustment and demonstrating substantial gains through simulations.

Contribution

The paper proposes a novel batch-adaptive stratification procedure with post-design adjustment, enhancing efficiency by utilizing covariate information and pairing strategies.

Findings

Significant improvements in precision and efficiency demonstrated in simulations.

Stratification complements regression adjustment, reducing adjustment errors.

Method performs well with both synthetic and real political science data.

Abstract

To increase statistical efficiency in a randomized experiment, researchers often use stratification (i.e., blocking) in the design stage. However, conventional practices of stratification fail to exploit valuable information about the predictive relationship between covariates and potential outcomes. In this paper, I introduce an adaptive stratification procedure for increasing statistical efficiency when some information is available about the relationship between covariates and potential outcomes. I show that, in a paired design, researchers can rematch observations across different batches. For inference, I propose a stratified estimator that allows for nonparametric covariate adjustment. I then discuss the conditions under which researchers should expect gains in efficiency from stratification. I show that stratification complements rather than substitutes for regression adjustment, insuring against adjustment error even when researchers plan to use covariate adjustment. To evaluate the performance of the method relative to common alternatives, I conduct simulations using both synthetic data and more realistic data derived from a political science experiment. Results demonstrate that the gains in precision and efficiency can be substantial.