Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance

TL;DR

This paper develops concentration bounds and asymptotic normality results for spectral projectors of sample covariance operators in high-dimensional settings, generalizing known models and aiding statistical inference in PCA.

Contribution

It provides sharp concentration bounds and asymptotic normality results for empirical spectral projectors in high-dimensional regimes with large sample size and effective rank.

Findings

Derived sharp concentration bounds for bilinear forms of spectral projectors.

Proved asymptotic normality of bilinear forms of empirical spectral projectors.

Established bounds on bias and methods for bias reduction.

Abstract

Let $X,X_1,\dots, X_n$ be i.i.d. Gaussian random variables with zero mean and covariance operator $\Sigma={\mathbb E}(X\otimes X)$ taking values in a separable Hilbert space ${\mathbb H}.$ Let $$ {\bf r}(\Sigma):=\frac{{\rm tr}(\Sigma)}{\|\Sigma\|_{\infty}} $$ be the effective rank of $\Sigma,$ ${\rm tr}(\Sigma)$ being the trace of $\Sigma$ and $\|\Sigma\|_{\infty}$ being its operator norm. Let $$\hat \Sigma_n:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n).$ The paper deals with a problem of estimation of spectral projectors of the covariance operator $\Sigma$ by their empirical counterparts, the spectral projectors of $\hat \Sigma_n$ (empirical spectral projectors). The focus is on the problems where both the sample size $n$ and the effective rank ${\bf r}(\Sigma)$ are large. This framework includes and generalizes well known high-dimensional spiked covariance models. Given a spectral projector $P_r$ corresponding to an eigenvalue $\mu_r$ of covariance operator $\Sigma$ and its empirical counterpart $\hat P_r,$ we derive sharp concentration bounds for bilinear forms of empirical spectral projector $\hat P_r$ in terms of sample size $n$ and effective dimension ${\bf r}(\Sigma).$ Building upon these concentration bounds, we prove the asymptotic normality of bilinear forms of random operators $\hat P_r -{\mathbb E}\hat P_r$ under the assumptions that $n\to \infty$ and ${\bf r}(\Sigma)=o(n).$ In a special case of eigenvalues of multiplicity one, these results are rephrased as concentration bounds and asymptotic normality for linear forms of empirical eigenvectors. Other results include bounds on the bias ${\mathbb E}\hat P_r-P_r$ and a method of bias reduction as well as a discussion of possible applications to statistical inference in high-dimensional principal component analysis.