Exact testing with random permutations

TL;DR

This paper establishes the theoretical foundation for exact permutation tests using random permutations, demonstrating their validity and extending their applicability beyond traditional assumptions.

Contribution

It provides an alternative proof of exactness for permutation tests with random permutations and extends the results to multiple testing procedures.

Findings

Proves the exactness of permutation tests with random permutations.

Introduces a new perspective by viewing the test as a conditional Monte Carlo test.

Extends the theoretical results to multiple testing procedures.

Abstract

When permutation methods are used in practice, often a limited number of random permutations are used to decrease the computational burden. However, most theoretical literature assumes that the whole permutation group is used, and methods based on random permutations tend to be seen as approximate. There exists a very limited amount of literature on exact testing with random permutations and only recently a thorough proof of exactness was given. In this paper we provide an alternative proof, viewing the test as a "conditional Monte Carlo test" as it has been called in the literature. We also provide extensions of the result. Importantly, our results can be used to prove properties of various multiple testing procedures based on random permutations.