Block what you can, except when you shouldn't
Nicole E. Pashley, Luke W. Miratrix

TL;DR
This paper examines the conditions under which blocking in experimental design improves or harms estimate precision, clarifying conflicting views and emphasizing that blocking generally offers benefits with minimal downsides.
Contribution
It reconciles conflicting perspectives on blocking, clarifies when it is beneficial or harmful, and discusses how assumptions and analysis methods influence outcomes.
Findings
Blocking usually improves precision with small potential costs.
Analyzing blocked experiments as completely randomized can sometimes reduce accuracy.
The benefits of blocking often outweigh the minimal risks.
Abstract
Several branches of the potential outcome causal inference literature have discussed the merits of blocking versus complete randomization. Some have concluded it can never hurt the precision of estimates, and some have concluded it can hurt. In this paper, we reconcile these apparently conflicting views, give a more thorough discussion of what guarantees no harm, and discuss how other aspects of a blocked design can cost, all in terms of precision. We discuss how the different findings are due to different sampling models and assumptions of how the blocks were formed. We also connect these ideas to common misconceptions, for instance showing that analyzing a blocked experiment as if it were completely randomized, a seemingly conservative method, can actually backfire in some cases. Overall, we find that blocking can have a price, but that this price is usually small and the potential…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
