Matrix Compression using the Nystro\"om Method

TL;DR

This paper extends the Nyström method to efficiently compute approximate SVD and EVD for general and square matrices, enabling effective matrix compression especially for matrices with rapidly decaying spectra.

Contribution

It introduces a novel approach to apply the Nyström method for matrix decompositions, including an algorithm for selecting optimal samples for improved accuracy.

Findings

Efficient approximate SVD and EVD computations in O(s^2(m+n)) and O(s^2 n) operations.

The method performs well on matrices with fast spectral decay.

A new sampling algorithm improves initial sample selection for better approximation.

Abstract

The Nystr\"{o}m method is routinely used for out-of-sample extension of kernel matrices. We describe how this method can be applied to find the singular value decomposition (SVD) of general matrices and the eigenvalue decomposition (EVD) of square matrices. We take as an input a matrix $M\in \mathbb{R}^{m\times n}$, a user defined integer $s\leq min(m,n)$ and $A_M \in \mathbb{R}^{s\times s}$, a matrix sampled from the columns and rows of $M$. These are used to construct an approximate rank-$s$ SVD of $M$ in $O\left(s^2\left(m+n\right)\right)$ operations. If $M$ is square, the rank-$s$ EVD can be similarly constructed in $O\left(s^2 n\right)$ operations. Thus, the matrix $A_M$ is a compressed version of $M$. We discuss the choice of $A_M$ and propose an algorithm that selects a good initial sample for a pivoted version of $M$. The proposed algorithm performs well for general matrices and kernel matrices whose spectra exhibit fast decay.