Improved Rates of Bootstrap Approximation for the Operator Norm: A Coordinate-Free Approach

TL;DR

This paper demonstrates that bootstrap methods can accurately approximate the distribution of the operator norm error of the sample covariance operator in high-dimensional Hilbert spaces, achieving near-parametric rates under certain eigenvalue decay conditions.

Contribution

It introduces a coordinate-free approach that improves the bootstrap approximation rates for the operator norm of covariance estimators, applicable to broader models including elliptical and Marčenko-Pastur types.

Findings

Bootstrap approximates the distribution at a rate of n^{-(β-1/2)/(2β+4+ε)}.

Achieves near n^{-1/2} rates for large β, surpassing previous n^{-1/6} rates.

Applicable to high-dimensional models with eigenvalues decaying as j^{-2β}.

Abstract

Let $\hat\Sigma=\frac{1}{n}\sum_{i=1}^n X_i\otimes X_i$ denote the sample covariance operator of centered i.i.d.~observations $X_1,\dots,X_n$ in a real separable Hilbert space, and let $\Sigma=\mathbb{E}(X_1\otimes X_1)$. The focus of this paper is to understand how well the bootstrap can approximate the distribution of the operator norm error $\sqrt n\|\hat\Sigma-\Sigma\|_{\text{op}}$, in settings where the eigenvalues of $\Sigma$ decay as $\lambda_j(\Sigma)\asymp j^{-2\beta}$ for some fixed parameter $\beta>1/2$. Our main result shows that the bootstrap can approximate the distribution of $\sqrt n\|\hat\Sigma-\Sigma\|_{\text{op}}$ at a rate of order $n^{-\frac{\beta-1/2}{2\beta+4+\epsilon}}$ with respect to the Kolmogorov metric, for any fixed $\epsilon>0$. In particular, this shows that the bootstrap can achieve near $n^{-1/2}$ rates in the regime of large $\beta$ -- which substantially improves on previous near $n^{-1/6}$ rates in the same regime. In addition to obtaining faster rates, our analysis leverages a fundamentally different perspective based on coordinate-free techniques. Moreover, our result holds in greater generality, and we propose a model that is compatible with both elliptical and Mar\v{c}enko-Pastur models in high-dimensional Euclidean spaces, which may be of independent interest.