Permutation Tests at Nonparametric Rates

TL;DR

This paper develops permutation tests for parameter equality that maintain correct size asymptotically and in finite samples, applicable to both parametric and nonparametric models, with broad practical and theoretical implications.

Contribution

It introduces a general framework for permutation tests that control size for parameters estimated at root-n or slower rates, extending classical methods to more complex settings.

Findings

Tests have correct asymptotic size and finite-sample exactness under distribution equality.

Simulations demonstrate good finite sample properties.

Applications to nonparametric hypothesis testing and empirical examples validate the approach.

Abstract

Classical two-sample permutation tests for equality of distributions have exact size in finite samples, but they fail to control size for testing equality of parameters that summarize each distribution. This paper proposes permutation tests for equality of parameters that are estimated at root-$n$ or slower rates. Our general framework applies to both parametric and nonparametric models, with two samples or one sample split into two subsamples. Our tests have correct size asymptotically while preserving exact size in finite samples when distributions are equal. They have no loss in local asymptotic power compared to tests that use asymptotic critical values. We propose confidence sets with correct coverage in large samples that also have exact coverage in finite samples if distributions are equal up to a transformation. We apply our theory to four commonly-used hypothesis tests of nonparametric functions evaluated at a point. Lastly, simulations show good finite sample properties, and two empirical examples illustrate our tests in practice.