Escaping Saddle Points in Ill-Conditioned Matrix Completion with a Scalable Second Order Method

TL;DR

This paper introduces a scalable second-order algorithm for low-rank matrix completion that effectively escapes saddle points, achieves near-optimal sample efficiency, and can handle highly ill-conditioned matrices.

Contribution

The paper presents a novel saddle-escaping Newton method that combines IRLS advantages with improved scalability and convergence properties for matrix completion.

Findings

Attains local quadratic convergence near the information limit.

Successfully completes matrices with condition numbers up to 10^10.

Outperforms state-of-the-art methods on ill-conditioned matrices.

Abstract

We propose an iterative algorithm for low-rank matrix completion that can be interpreted as both an iteratively reweighted least squares (IRLS) algorithm and a saddle-escaping smoothing Newton method applied to a non-convex rank surrogate objective. It combines the favorable data efficiency of previous IRLS approaches with an improved scalability by several orders of magnitude. Our method attains a local quadratic convergence rate already for a number of samples that is close to the information theoretical limit. We show in numerical experiments that unlike many state-of-the-art approaches, our approach is able to complete very ill-conditioned matrices with a condition number of up to $10^{10}$ from few samples.