Does Rerandomization Help Beyond Covariate Adjustment? A Review and Guide for Theory and Practice

TL;DR

This paper reviews rerandomization in experimental design, examining its theoretical foundations, finite-sample performance, and practical benefits, especially for small samples, showing it enhances estimator precision and robustness.

Contribution

It provides a comprehensive review, simulation analysis, and practical guidance on rerandomization, highlighting its benefits beyond asymptotic precision, particularly in finite samples.

Findings

Rerandomization improves finite-sample estimator precision.

It increases robustness and reliability of causal estimates.

Benefits are especially notable in smaller samples.

Abstract

Rerandomization is a modern experimental design technique that repeatedly randomizes treatment assignments until covariates are deemed balanced between treatment groups. This enhances the precision and coherence of causal effect estimators, mitigates false discoveries from p-hacking, and increases statistical power. Recent work suggests that balancing covariates via rerandomization does not alter the asymptotic precision of covariate-adjusted estimators, thereby making it unclear whether rerandomization is worthwhile if adjusted estimators are used. However, these results have two key caveats. First, these results are asymptotic, leaving finite sample performance unknown. Second, these results focus on precision, while other potential benefits, such as increased coherence among flexible estimators, remain understudied. Hence, in this paper we provide three main contributions: (i) a comprehensive review of the rerandomization literature, covering historical foundations, theoretical developments, and recent methodological advancements, (ii) an extensive simulation study examining finite-sample performance, and (iii) a practical guide for practitioners. Our study compares precision, coherence, power, and coverage of various estimators under rerandomization versus complete randomization. We find rerandomization to be a complementary design strategy that enhances the precision, robustness, and reliability of causal effect estimators, especially for smaller sample sizes.