A Comparison of Methods of Inference in Randomized Experiments from a Restricted Set of Allocations

TL;DR

This paper compares various inference methods in rerandomized experiments, showing some outperform existing approaches especially in small to moderate sample sizes through extensive simulations.

Contribution

It introduces and evaluates alternative inference methods for rerandomization, demonstrating improvements over previous approaches in certain sample size conditions.

Findings

Some methods outperform Li et al. (2018) in small or moderate samples.

Alternative methods provide less conservative inference after rerandomization.

Simulation results guide choice of inference method based on sample size.

Abstract

Rerandomization is a strategy of increasing efficiency as compared to complete randomization. The idea with rerandomization is that of removing allocations with imbalance in the observed covariates and then randomizing within the set of allocations with balance in these covariates. Standard asymptotic inference based on mean difference estimator is however conservative after rerandomization. Given a Mahalanobis distance criterion for removing imbalanced allocations, Li et al. (2018) derived the asymptotic distribution of the mean difference estimator and suggested a consistent estimator of its variance. This paper discusses several alternative methods of inference under rerandomization, and compare their performance with that of the method in Li et al. (2018) through a large Monte Carlo simulation. We conclude that some of the methods work better for small or moderate sample sized experiments than the method in Li et al. (2018).