Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions

TL;DR

This paper reviews and extends probabilistic algorithms for low-rank matrix approximations, highlighting their efficiency, accuracy, and suitability for large-scale data analysis compared to classical methods.

Contribution

It introduces a modular framework for randomized matrix decompositions that improves performance and robustness over traditional techniques.

Findings

Randomized algorithms outperform classical methods in speed and accuracy.

The approach effectively handles large-scale data sets.

Extensive experiments validate the method's robustness and efficiency.

Abstract

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed---either explicitly or implicitly---to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, speed, and robustness. These claims are supported by extensive numerical experiments and a detailed error analysis.