Optimal Eigenvalue Shrinkage in the Semicircle Limit

TL;DR

This paper develops optimal eigenvalue shrinkage methods for high-dimensional covariance estimation under disproportional growth regimes, providing unified rules that are asymptotically optimal across different data aspect ratios.

Contribution

It introduces new closed-form optimal shrinkage and thresholding rules for the spiked covariance model in disproportional asymptotic regimes, extending previous proportional-growth analyses.

Findings

Optimal shrinkage rules outperform standard estimators.

Unified rules are framework agnostic, applicable to various growth regimes.

Significant performance improvements demonstrated through theoretical analysis.

Abstract

Modern datasets are trending towards ever higher dimension. In response, recent theoretical studies of covariance estimation often assume the proportional-growth asymptotic framework, where the sample size $n$ and dimension $p$ are comparable, with $n, p \rightarrow \infty $ and $\gamma_n = p/n \rightarrow \gamma > 0$. Yet, many datasets -- perhaps most -- have very different numbers of rows and columns. We consider instead the disproportional-growth asymptotic framework, where $n, p \rightarrow \infty$ and $\gamma_n \rightarrow 0$ or $\gamma_n \rightarrow \infty$. Either disproportional limit induces novel behavior unseen within previous proportional and fixed-$p$ analyses. We study the spiked covariance model, with theoretical covariance a low-rank perturbation of the identity. For each of 15 different loss functions, we exhibit in closed form new optimal shrinkage and thresholding rules. Our optimal procedures demand extensive eigenvalue shrinkage and offer substantial performance benefits over the standard empirical covariance estimator. Practitioners may ask whether to view their data as arising within (and apply the procedures of) the proportional or disproportional frameworks. Conveniently, it is possible to remain {\it framework agnostic}: one unified set of closed-form shrinkage rules (depending only on the aspect ratio $\gamma_n$ of the given data) offers full asymptotic optimality under either framework. At the heart of the phenomena we explore is the spiked Wigner model, in which a low-rank matrix is perturbed by symmetric noise. Exploiting a connection to the spiked covariance model as $\gamma_n \rightarrow 0$, we derive optimal eigenvalue shrinkage rules for estimation of the low-rank component, of independent and fundamental interest.