Efficient Bounds and Estimates for Canonical Angles in Randomized Subspace Approximations

TL;DR

This paper develops practical, efficiently computable bounds and estimates for the accuracy of singular vectors in randomized subspace approximations, specifically focusing on canonical angles, enhancing understanding of their precision without prior knowledge.

Contribution

It introduces new bounds and estimates for canonical angles in randomized SVD, which are asymptotically tight and do not require prior knowledge of true singular subspaces.

Findings

Bounds are asymptotically tight under moderate oversampling.

Bounds can be computed efficiently a priori or a posteriori.

Numerical experiments confirm the empirical effectiveness of the bounds.

Abstract

Randomized subspace approximation with "matrix sketching" is an effective approach for constructing approximate partial singular value decompositions (SVDs) of large matrices. The performance of such techniques has been extensively analyzed, and very precise estimates on the distribution of the residual errors have been derived. However, our understanding of the accuracy of the computed singular vectors (measured in terms of the canonical angles between the spaces spanned by the exact and the computed singular vectors, respectively) remains relatively limited. In this work, we present practical bounds and estimates for canonical angles of randomized subspace approximation that can be computed efficiently either a priori or a posteriori, without assuming prior knowledge of the true singular subspaces. Under moderate oversampling in the randomized SVD, our prior probabilistic bounds are asymptotically tight and can be computed efficiently, while bringing a clear insight into the balance between oversampling and power iterations given a fixed budget on the number of matrix-vector multiplications. The numerical experiments demonstrate the empirical effectiveness of these canonical angle bounds and estimates on different matrices under various algorithmic choices for the randomized SVD.