# Iterative Refinement and Oversampling for Low Rank Approximation

## TL;DR

This paper extends iterative refinement to low rank matrix approximation, linking it to oversampling techniques, and demonstrates that using sparse sketches enables very fast, low-cost approximations with near-optimal accuracy for certain matrices.

## Contribution

It introduces a novel iterative refinement approach for low rank approximation, connecting it to oversampling, and shows how sparse sketch matrices enable efficient, very low rank approximations with improved accuracy.

## Key findings

- Sparse sketch matrices enable sublinear cost low rank approximation.
- Oversampling improves accuracy to near-optimal levels.
- The method is effective for matrices with fast decaying singular values.

## Abstract

Iterative refinement is particularly popular for numerical solution of linear systems of equations. We extend it to Low Rank Approximation of a matrix (LRA) and observe close link of the resulting algorithm to oversampling techniques, commonly used in randomized LRA algorithms. We elaborate upon this link and revisit oversampling and some efficient randomized LRA algorithms. Applied with sparse sketch matrices they run significantly faster and in particular yield Very Low Rank Approximation (VLRA) at sublinear cost, using much fewer scalars and flops than the input matrix has entries. This is achieved at the price of deterioration of output accuracy, but according to our formal and empirical study subsequent oversampling improves accuracy to near-optimal level under the spectral norm for a large sub-class of matrices with fast decaying spectra of singular values.