Randomization Inference: Theory and Applications

TL;DR

This paper reviews the theory and applications of randomization inference, highlighting conditions for exact and asymptotic error control, and demonstrating its use in various statistical testing scenarios.

Contribution

It provides a comprehensive overview of randomization tests, including their theoretical foundations, conditions for validity, and practical applications across different statistical methods.

Findings

Randomization tests achieve exact finite-sample error control under the randomization hypothesis.

They are often asymptotically valid and efficient even when conditions for exact control are not met.

Constructing asymptotically pivotal test statistics can restore validity when tests fail to control Type I error.

Abstract

We review approaches to statistical inference based on randomization. Permutation tests are treated as an important special case. Under a certain group invariance property, referred to as the ``randomization hypothesis,'' randomization tests achieve exact control of the Type I error rate in finite samples. Although this unequivocal precision is very appealing, the range of problems that satisfy the randomization hypothesis is somewhat limited. We show that randomization tests are often asymptotically, or approximately, valid and efficient in settings that deviate from the conditions required for finite-sample error control. When randomization tests fail to offer even asymptotic Type 1 error control, their asymptotic validity may be restored by constructing an asymptotically pivotal test statistic. Randomization tests can then provide exact error control for tests of highly structured hypotheses with good performance in a wider class of problems. We give a detailed overview of several prominent applications of randomization tests, including two-sample permutation tests, regression, and conformal inference.