Nearly Random Designs with Greatly Improved Balance

TL;DR

This paper introduces a greedy algorithm for experimental design that achieves near-perfect covariate balance while maintaining randomness, improving upon traditional randomization methods especially for small to moderate sample sizes.

Contribution

The authors develop a novel greedy algorithm that produces highly balanced experimental designs with minimal departure from randomness, extending to multiple covariates and providing an open-source R package.

Findings

Designs achieve $O_p(n^{-3})$ covariate balance for one covariate.

Assignments are nearly as random as complete randomization based on entropy and standard error measures.

Simulations demonstrate improved balance without sacrificing randomness.

Abstract

We present a new experimental design procedure that divides a set of experimental units into two groups so that the two groups are balanced on a prespecified set of covariates and being almost as random as complete randomization. Under complete randomization, the difference in covariate balance as measured by the standardized average between treatment and control will be $O_p(n^{-1/2})$. If the sample size is not too large this may be material. In this article, we present an algorithm which greedily switches assignment pairs. Resultant designs produce balance of the much lower order $O_p(n^{-3})$ for one covariate. However, our algorithm creates assignments which are, strictly speaking, non-random. We introduce two metrics which capture departures from randomization: one in the style of entropy and one in the style of standard error and demonstrate our assignments are nearly as random as complete randomization in terms of both measures. The results are extended to more than one covariate, simulations are provided to illustrate the results and statistical inference under our design is discussed. We provide an open source R package available on CRAN called GreedyExperimentalDesign which generates designs according to our algorithm.