A powerful goodness-of-fit test using the probability integral transform of order statistics

TL;DR

PITOS is a new goodness-of-fit test leveraging the probability integral transform of order statistics, offering higher power against local deviations from the null hypothesis while maintaining computational efficiency and broad applicability.

Contribution

The paper introduces PITOS, a novel GoF test based on the probability integral transform of order statistics, with improved power and efficiency over existing tests.

Findings

PITOS maintains valid Type I error control.

PITOS outperforms popular GoF tests on local departure distributions.

PITOS has an efficient $O(n \, \log n)$ runtime.

Abstract

Goodness-of-fit (GoF) tests are a fundamental component of statistical practice, essential for checking model assumptions and testing scientific hypotheses. Despite their widespread use, popular GoF tests exhibit surprisingly low statistical power against substantial departures from the null hypothesis. To address this, we introduce PITOS, a novel GoF test based on applying the probability integral transform (PIT) to the $j$th order statistic (OS) given the $i$th order statistic for selected pairs $i,j$. Under the null, for any pair $i,j$, this yields a $\mathrm{Uniform}(0,1)$ random variable, which we map to a p-value via $u\mapsto 2\min(u, 1-u)$. We compute these p-values for a structured collection of pairs $i,j$ generated via a discretized transformed Halton sequence, and aggregate them using the Cauchy combination technique to obtain the PITOS p-value. Our method maintains approximately valid Type I error control, has an efficient $O(n \log n)$ runtime, and can be used with any null distribution via the Rosenblatt transform. In empirical demonstrations, we find that PITOS has much higher power than popular GoF tests on distributions characterized by local departures from the null, while maintaining competitive power across all distributions tested.