CUR Low Rank Approximation of a Matrix at Sublinear Cost

TL;DR

This paper discusses methods for low rank matrix approximation at sublinear computational cost, highlighting classes of matrices where such approximations are feasible and effective in practice.

Contribution

It identifies classes of matrices suitable for sublinear cost CUR low rank approximations and presents empirically accurate algorithms for large matrices.

Findings

CUR LRAs can be computed at sublinear cost for certain matrix classes.

Cross-Approximation iterations effectively produce accurate LRAs.

Some techniques are of independent interest for matrix approximation methods.

Abstract

Low rank approximation of a matrix (hereafter LRA) is a highly important area of Numerical Linear and Multilinear Algebra and Data Mining and Analysis. One can operate with an LRA at sublinear cost -- by using much fewer memory cells and flops than an input matrix M has entries. For worst case inputs one cannot compute even a reasonably close LRA at sublinear cost, but in computational practice accurate LRAs, even in their memory efficient form of CUR LRAs, are routinely obtained at sublinear cost for large and important classes of matrices, in particular by means of Cross-Approximation iterations, which specialize Alternating Direction techniques to LRA. We identify some classes of matrices for which CUR LRA are computed at sublinear cost as well as some sublinear cost LRA algorithms that are empirically accurate for large classes of inputs. Some of our techniques and concepts can be of independent interests.