On a Probability Inequality for Order Statistics with Applications to Bootstrap, Conformal Prediction, and more

TL;DR

This paper extends a probability inequality for order statistics to approximate scenarios involving dependence and distributional differences, with applications to bootstrap, conformal prediction, and rank tests.

Contribution

It develops approximate versions of a probability inequality under violations of independence and identical distribution assumptions, enabling new inference methods.

Findings

Provides an inequality that holds approximately under dependence and distributional differences.

Shows the inequality's use in proving bootstrap, conformal prediction, and permutation test validity.

Introduces a 'cheap bootstrap' method suitable for high-dimensional data.

Abstract

``Behind every limit theorem, there is an inequality'' said Kolmogorov. We say ``for every inequality, there is an approximate inequality under approximate regularity conditions.'' Suppose $X, X'$ are independent and identically distributed random variables. Then $X \le X'$ with a probability of at least $1/2$, irrespective of the underlying (common) distribution. One can ask what happens to the probability if $X, X'$ are independent but not identically distributed. It should be approximately $1/2$ if the distributions are approximately equal. Similarly, what if the random variables are dependent? It should, again, be approximately $1/2$ if the random variables are approximately independent. We explore an extension of this probability inequality involving order statistics and develop approximate versions of such an inequality under violations of independence and identical distribution assumptions. We further show that this inequality can be used as a basis to prove asymptotic validity of bootstrap/subsampling, finite-sample validity of conformal prediction, permutation tests, and asymptotic validity of rank tests without group invariance. Specifically, in the context of resampling inference, our results can be seen as a finite-sample instantiation of some results by Peter Hall and yield an alternative ``cheap bootstrap'' that applies to high-dimensional data.