Block Designs that Provide Optimal Power in the Cochran-Mantel-Haenszel Test

TL;DR

This paper analyzes the asymptotic power of the Cochran-Mantel-Haenszel test under Fisher's blocking design, identifying optimal blocking strategies and practical recommendations for various sample sizes and covariate complexities.

Contribution

It establishes the asymptotic optimality of certain blocking designs under local alternatives and provides practical guidance for design choices in experiments.

Findings

Blocking designs satisfying a balance condition are asymptotically optimal.

Pairwise matching design meets the balance condition under mild assumptions.

In small samples, designs with fewer blocks become more effective.

Abstract

We consider the asymptotic power performance under local alternatives of the Cochran-Mantel-Haenszel test. Our setting is non-traditional: we investigate randomized experiments that assign subjects via Fisher's blocking design. We show that blocking designs that satisfy a certain balance condition are asymptotically optimal. When the potential outcomes can be ordered, the balance condition is met for all blocking designs with number of blocks going to infinity. More generally, we prove that the pairwise matching design of Greevy et al. (2004) satisfies the balance condition under mild assumptions. In smaller sample sizes, we show a second order effect becomes operational thereby making blocking designs with a smaller number optimal. In practical settings with many covariates, we recommend pairwise matching for its ability to approximate the balance condition.