Randomized low-rank approximations beyond Gaussian random matrices

TL;DR

This paper extends the theoretical analysis of randomized low-rank approximation methods to include various classes of non-Gaussian random matrices, providing insights into their effectiveness and requirements.

Contribution

It introduces a novel interpretation of approximation error and offers bounds for different random matrix classes beyond Gaussian, broadening the understanding of randomized low-rank approximation.

Findings

Bounds on approximation error for sub-Gaussian and bounded matrices

Insights into minimal sample size for effective approximation

Numerical validation with synthetic and real data

Abstract

This paper expands the analysis of randomized low-rank approximation beyond the Gaussian distribution to four classes of random matrices: (1) independent sub-Gaussian entries, (2) independent sub-Gaussian columns, (3) independent bounded columns, and (4) independent columns with bounded second moment. Using a novel interpretation of the low-rank approximation error involving sample covariance matrices, we provide insight into the requirements of a \textit{good random matrix} for the purpose of randomized low-rank approximation. Although our bounds involve unspecified absolute constants (a consequence of the underlying non-asymptotic theory of random matrices), they allow for qualitative comparisons across distributions. The analysis offers some details on the minimal number of samples (the number of columns $\ell$ of the random matrix $\boldsymbol\Omega$) and the error in the resulting low-rank approximation. We illustrate our analysis in the context of the randomized subspace iteration method as a representative algorithm for low-rank approximation, however, all the results are broadly applicable to other low-rank approximation techniques. We conclude our discussion with numerical examples using both synthetic and real-world test matrices.