Faster Low-rank Approximation using Adaptive Gap-based Preconditioning

TL;DR

This paper introduces a faster algorithm for low-rank matrix approximation that adapts to spectral gaps, achieving near-linear time complexity under certain spectral gap conditions.

Contribution

It presents a novel adaptive gap-based preconditioning method that significantly improves the efficiency of low-rank approximation algorithms.

Findings

Achieves near-linear time for constant rank and stable spectral gaps.

Provides a new analysis linking spectral gaps to algorithmic complexity.

Demonstrates improved performance over existing methods in specific spectral regimes.

Abstract

We propose a method for rank $k$ approximation to a given input matrix $X \in \mathbb{R}^{d \times n}$ which runs in time \[ \tilde{O} \left(d ~\cdot~ \min\left\{n + \tilde{sr}(X) \,G^{-2}_{k,p+1}\ ,\ n^{3/4}\, \tilde{sr}(X)^{1/4} \,G^{-1/2}_{k,p+1} \right\} ~\cdot~ \text{poly}(p)\right) ~, \] where $p>k$, $\tilde{sr}(X)$ is related to stable rank of $X$, and $G_{k,p+1} = \frac{\sigma_k-\sigma_p}{\sigma_k}$ is the multiplicative gap between the $k$-th and the $(p+1)$-th singular values of $X$. In particular, this yields a linear time algorithm if the gap is at least $1/\sqrt{n}$ and $k,p,\tilde{sr}(X)$ are constants.