Simple and practical algorithms for $\ell_p$-norm low-rank approximation

TL;DR

This paper introduces practical, non-convex, gradient-based algorithms for entrywise  low-rank approximation that are easy to implement, achieve near-optimal solutions, and are supported by theoretical guarantees.

Contribution

It presents new practical algorithms for  low-rank approximation with theoretical approximation guarantees, bridging the gap between empirical performance and theoretical analysis.

Findings

Algorithms attain -ps approximation within polynomial time.

Proposed methods outperform existing state-of-the-art in speed and accuracy.

Theoretical analysis clarifies the problem parameters needed for guarantees.

Abstract

We propose practical algorithms for entrywise $\ell_p$-norm low-rank approximation, for $p = 1$ or $p = \infty$. The proposed framework, which is non-convex and gradient-based, is easy to implement and typically attains better approximations, faster, than state of the art. From a theoretical standpoint, we show that the proposed scheme can attain $(1 + \varepsilon)$-OPT approximations. Our algorithms are not hyperparameter-free: they achieve the desiderata only assuming algorithm's hyperparameters are known a priori---or are at least approximable. I.e., our theory indicates what problem quantities need to be known, in order to get a good solution within polynomial time, and does not contradict to recent inapproximabilty results, as in [46].