The performance and efficiency of Threshold Blocking

TL;DR

This paper compares threshold blocking and fixed-sized blocking methods for experimental treatment assignment, showing threshold blocking's superiority in most cases, especially when covariates strongly predict outcomes.

Contribution

It provides a theoretical and empirical analysis demonstrating the advantages of threshold blocking over fixed-sized blocking in reducing variance.

Findings

Threshold blocking always finds a weakly better grouping than fixed-sized blocking.

Threshold blocking performs better when covariates are highly predictive of outcomes.

Fixed-sized blocking can outperform threshold blocking when the objective function is unknown.

Abstract

A common method to reduce the uncertainty of causal inferences from experiments is to assign treatments in fixed proportions within groups of similar units: blocking. Previous results indicate that one can expect substantial reductions in variance if these groups are formed so to contain exactly as many units as treatment conditions. This approach can be contrasted to threshold blocking which, instead of specifying a fixed size, requires that the groups contain a minimum number of units. In this paper, I investigate the advantages of respective method. In particular, I show that threshold blocking is superior to fixed-sized blocking in the sense that it always finds a weakly better grouping for any objective and sample. However, this does not necessarily hold when the objective function of the blocking problem is unknown, and a fixed-sized design can perform better in that case. I specifically examine the factors that govern how the methods perform in the common situation where the objective is to reduce the estimator's variance, but where groups are constructed based on covariates. This reveals that the relative performance of threshold blocking improves when the covariates become more predictive of the outcome.