Low Rank Approximation at Sublinear Cost

TL;DR

This paper investigates the feasibility of computing Low Rank Approximation (LRA) of matrices at sublinear cost, demonstrating that accelerated algorithms perform well on average but may fail on worst-case matrices, with practical accuracy confirmed through tests.

Contribution

The paper accelerates popular random sketching LRA algorithms to run at sublinear cost and analyzes their expected accuracy under probabilistic models, showing they succeed for most inputs.

Findings

Accelerated algorithms achieve sublinear cost for LRA.

Expected error norms remain within a reasonable factor of optimal.

Algorithms perform accurately on synthetic and real-world matrices in tests.

Abstract

Low Rank Approximation (LRA) of a matrix is a hot research subject, fundamental for Matrix and Tensor Computations and Big Data Mining and Analysis. Computations with low rank matrices can be performed at sublinear cost -- by using much fewer floating-point operations (flops) than an input matrix has entries, but can we compute LRA at sublinear cost? This is routinely done in computational practice for a large class of inputs, even though any sublinear cost LRA algorithm fails most miserably on worst case matrices. To provide insight into this controversy we first accelerate some popular near-optimal random sketching LRA algorithms -- to run them at sublinear cost. Then we define two probabilistic structures in the space of input matrices and estimate that the expected spectral and Frobenius error norms for the output LRA of the accelerated algorithms stay within a reasonable factor from their optima under both models, and so these sublinear cost algorithms only fail for a very narrow input class. Our upper estimates for their output accuracy are still quite high, but under some additional semi-heuristic amendments the algorithms have consistently output accurate LRA of various synthetic and real-world matrices in our numerical tests.