Exact and asymptotically robust permutation tests
EunYi Chung, Joseph P. Romano

TL;DR
This paper develops permutation tests that are exactly level when distributions are identical and asymptotically valid for parameter comparisons, applicable to various parameters and sample sizes.
Contribution
It introduces a general permutation testing framework that ensures exact level in finite samples and asymptotic correctness for parameter-based hypotheses.
Findings
Permutation tests are exact when distributions are identical.
The proposed tests are asymptotically valid for parameter nulls.
Simulation studies support the theoretical results.
Abstract
Given independent samples from P and Q, two-sample permutation tests allow one to construct exact level tests when the null hypothesis is P=Q. On the other hand, when comparing or testing particular parameters of P and Q, such as their means or medians, permutation tests need not be level , or even approximately level in large samples. Under very weak assumptions for comparing estimators, we provide a general test procedure whereby the asymptotic validity of the permutation test holds while retaining the exact rejection probability in finite samples when the underlying distributions are identical. The ideas are broadly applicable and special attention is given to the k-sample problem of comparing general parameters, whereby a permutation test is constructed which is exact level under the hypothesis of identical distributions, but has…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
