Gaussian and Bootstrap Approximations for Suprema of Empirical Processes

TL;DR

This paper introduces new non-asymptotic Gaussian approximation methods for the supremum of empirical processes, applicable even when the function class varies with sample size and lacks Donsker properties, enabling advanced high-dimensional inference.

Contribution

It provides a simplified, broadly applicable Gaussian approximation framework that relaxes previous entropy and variance conditions, along with a novel bootstrap procedure based on Gaussian process decomposition.

Findings

Applicable to high-dimensional inference problems

Enables bootstrap procedures for empirical process suprema

Demonstrated on covariance matrices and confidence bands

Abstract

In this paper we develop non-asymptotic Gaussian approximation results for the sampling distribution of suprema of empirical processes when the indexing function class $\mathcal{F}_n$ varies with the sample size $n$ and may not be Donsker. Prior approximations of this type required upper bounds on the metric entropy of $\mathcal{F}_n$ and uniform lower bounds on the variance of $f \in \mathcal{F}_n$ which, both, limited their applicability to high-dimensional inference problems. In contrast, the results in this paper hold under simpler conditions on boundedness, continuity, and the strong variance of the approximating Gaussian process. The results are broadly applicable and yield a novel procedure for bootstrapping the distribution of empirical process suprema based on the truncated Karhunen-Lo{\`e}ve decomposition of the approximating Gaussian process. We demonstrate the flexibility of this new bootstrap procedure by applying it to three fundamental problems in high-dimensional statistics: simultaneous inference on parameter vectors, inference on the spectral norm of covariance matrices, and construction of simultaneous confidence bands for functions in reproducing kernel Hilbert spaces.