Matrix perturbation analysis of methods for extracting singular values from approximate singular subspaces

TL;DR

This paper analyzes the accuracy of methods for extracting singular values from approximate subspaces, highlighting the superior performance of the generalized Nyström approximation through a novel matrix perturbation framework.

Contribution

It introduces a new perturbation analysis for singular value approximation methods, especially demonstrating the effectiveness of the generalized Nyström approach over classical techniques.

Findings

Generalized Nyström achieves higher accuracy than classical methods.

Derived sharp bounds accurately predict approximation errors.

Extended analysis to block tridiagonal matrices and practical a-posteriori bounds.

Abstract

Given (orthonormal) approximations $\tilde{U}$ and $\tilde{V}$ to the left and right subspaces spanned by the leading singular vectors of a matrix $A$, we discuss methods to approximate the leading singular values of $A$ and study their accuracy. In particular, we focus our analysis on the generalized Nystr\"om approximation, as surprisingly, it is able to obtain significantly better accuracy than classical methods, namely Rayleigh-Ritz and (one-sided) projected SVD. A key idea of the analysis is to view the methods as finding the exact singular values of a perturbation of $A$. In this context, we derive a matrix perturbation result that exploits the structure of such $2\times2$ block matrix perturbation. Furthermore, we extend it to block tridiagonal matrices. We then obtain bounds on the accuracy of the extracted singular values. This leads to sharp bounds that predict well the approximation error trends and explain the difference in the behavior of these methods. Finally, we present an approach to derive an a-posteriori version of those bounds, which are more amenable to computation in practice.