More Power by using Fewer Permutations

TL;DR

This paper demonstrates that using smaller permutation subgroups can significantly increase statistical power and reduce computational costs in permutation tests, especially in high-dimensional settings.

Contribution

It challenges the conventional belief that larger permutation sets are always better, showing that tiny subgroups can offer more power and efficiency.

Findings

Tiny permutation subgroups can outperform full sets in power.

Using smaller groups reduces computational costs.

High-dimensional cases benefit most from this approach.

Abstract

It is conventionally believed that a permutation test should ideally use all permutations. If this is computationally unaffordable, it is believed one should use the largest affordable Monte Carlo sample or (algebraic) subgroup of permutations. We challenge this belief by showing we can sometimes obtain dramatically more power by using a tiny subgroup. As the subgroup is tiny, this simultaneously comes at a much lower computational cost. We exploit this to improve the popular permutation-based Westfall & Young MaxT multiple testing method. We study the relative efficiency in a Gaussian location model, and find the largest gain in high dimensions.