Subspace Iteration Randomization and Singular Value Problems

TL;DR

This paper introduces a new error analysis for randomized subspace iteration algorithms that efficiently produce highly accurate low-rank matrix approximations, especially for matrices with rapidly decaying singular values, with high probability.

Contribution

It provides a novel probabilistic error analysis for randomized subspace iteration methods, demonstrating their effectiveness in computing accurate low-rank approximations and singular values.

Findings

High-probability accuracy in low-rank approximations

Effective rank-revealing properties of randomized algorithms

Reliable estimation of matrix 2-norms using rank-1 approximations

Abstract

A classical problem in matrix computations is the efficient and reliable approximation of a given matrix by a matrix of lower rank. The truncated singular value decomposition (SVD) is known to provide the best such approximation for any given fixed rank. However, the SVD is also known to be very costly to compute. Among the different approaches in the literature for computing low-rank approximations, randomized algorithms have attracted researchers' recent attention due to their surprising reliability and computational efficiency in different application areas. Typically, such algorithms are shown to compute with very high probability low-rank approximations that are within a constant factor from optimal, and are known to perform even better in many practical situations. In this paper, we present a novel error analysis that considers randomized algorithms within the subspace iteration framework and show with very high probability that highly accurate low-rank approximations as well as singular values can indeed be computed quickly for matrices with rapidly decaying singular values. Such matrices appear frequently in diverse application areas such as data analysis, fast structured matrix computations and fast direct methods for large sparse linear systems of equations and are the driving motivation for randomized methods. Furthermore, we show that the low-rank approximations computed by these randomized algorithms are actually rank-revealing approximations, and the special case of a rank-1 approximation can also be used to correctly estimate matrix 2-norms with very high probability. Our numerical experiments are in full support of our conclusions.