Prepivoted permutation tests

TL;DR

This paper introduces prepivoted permutation tests that are exact under the null hypothesis of distribution equality and asymptotically valid for parameter equality, improving test accuracy using transformation techniques.

Contribution

It proposes a general prepivoting framework for permutation tests, enhancing their robustness and accuracy through asymptotic and bootstrap methods.

Findings

Permutation tests become exact under distribution equality.

Bootstrap prepivoting improves error rates in parameter testing.

Simulation confirms the method's versatility and validity.

Abstract

We present a general approach to constructing permutation tests that are both exact for the null hypothesis of equality of distributions and asymptotically correct for testing equality of parameters of distributions while allowing the distributions themselves to differ. These robust permutation tests transform a given test statistic by a consistent estimator of its limiting distribution function before enumerating its permutation distribution. This transformation, known as prepivoting, aligns the unconditional limiting distribution for the test statistic with the probability limit of its permutation distribution. Through prepivoting, the tests permute one minus an asymptotically valid $p$-value for testing the null of equality of parameters. We describe two approaches for prepivoting within permutation tests, one directly using asymptotic normality and the other using the bootstrap. We further illustrate that permutation tests using bootstrap prepivoting can provide improvements to the order of the error in rejection probability relative to competing transformations when testing equality of parameters, while maintaining exactness under equality of distributions. Simulation studies highlight the versatility of the proposal, illustrating the restoration of asymptotic validity to a wide range of permutation tests conducted when only the parameters of distributions are equal.