Accuracy of empirical projections of high-dimensional Gaussian matrices

TL;DR

This paper provides non-asymptotic bounds on the accuracy of reduced-rank projections of high-dimensional Gaussian matrices with a deterministic component, with implications for low-rank matrix recovery.

Contribution

It introduces a novel approach that avoids perturbation theory, providing sharp non-asymptotic bounds involving singular values, applicable even with singular value multiplicities.

Findings

Derived universal upper and lower bounds for projection accuracy.

Characterized matrix prototypes with favorable and unfavorable approximation properties.

Discussed implications for statistical estimation and low-rank matrix recovery.

Abstract

Let $X=C+\mathrm{E}$ with a deterministic matrix $C\in\R^{M\times M}$ and $\mathrm{E}$ some centered Gaussian $M\times M$-matrix whose entries are independent with variance $\sigma^2$. In the present work, the accuracy of reduced-rank projections of $X$ is studied. Non-asymptotic universal upper and lower bounds are derived, and favorable and unfavorable prototypes of matrices $C$ in terms of the accuracy of approximation are characterized. The approach does not involve analytic perturbation theory of linear operators and allows for multiplicities in the singular value spectrum. Our main result is some general non-asymptotic upper bound on the accuracy of approximation which involves explicitly the singular values of $C$, and which is shown to be sharp in various regimes of $C$. The results are accompanied by lower bounds under diverse assumptions. Consequences on statistical estimation problems, in particular in the recent area of low-rank matrix recovery, are discussed.