Moment bounds for condition numbers and singular values of high-dimensional Gaussian random matrices: Applications and limitations

TL;DR

This paper derives non-asymptotic moment bounds for the extreme singular values and condition numbers of high-dimensional Gaussian matrices, enabling improved risk analysis in statistical estimation and highlighting limitations for sub-Gaussian matrices.

Contribution

It establishes new non-asymptotic moment bounds for Gaussian matrices' singular values and condition numbers, filling a key gap in high-dimensional spectral analysis.

Findings

Derived explicit moment bounds for Gaussian matrices.

Applied bounds to high-dimensional regression and covariance estimation.

Showed limitations for extending bounds to sub-Gaussian matrices.

Abstract

Spectral properties of Gram matrices are central to high dimensional asymptotic analyses of statistical estimators in regression and covariance estimation. These properties, in turn, depend critically on the extreme singular values and condition numbers of Gaussian random matrices. For many applications, sharp positive and negative moment bounds for these quantities are required to control expected prediction risk and related performance metrics. Although extensive work provides concentration and tail bounds for extreme singular values of Gaussian random matrices, these results do not readily yield the moment bounds needed in such analyses. Motivated by this gap, we establish non asymptotic moment bounds for arbitrary positive moments of the largest singular value and arbitrary negative moments of the smallest singular value, and uniform bounds for arbitrary positive moments of the condition number of high dimensional Gaussian random matrices. We demonstrate the utility of these bounds by applying them to derive explicit risk guarantees in high dimensional regression and covariance estimation, as well as to obtain bounds on the mean iteration complexity of gradient descent for solving Gram linear systems. Finally, we present counterexamples demonstrating that the positive condition number moment bounds and negative smallest singular value moment bounds cannot, in general, be extended to the broader class of sub Gaussian random matrices.