Noisy Low-rank Matrix Optimization: Geometry of Local Minima and Convergence Rate

TL;DR

This paper develops a new theoretical framework for low-rank matrix optimization under noisy data, extending the understanding of local minima and convergence rates with less restrictive conditions than previous studies.

Contribution

It introduces a novel mathematical approach that relaxes the RIP constant requirements, providing comprehensive analysis of the optimization landscape under noise.

Findings

RIP constant less than 1/3 ensures spurious minima are near the ground truth

Approximate solutions can be found in polynomial time using strict saddle properties

The analysis applies to general low-rank problems with random corruptions

Abstract

This paper is concerned with low-rank matrix optimization, which has found a wide range of applications in machine learning. This problem in the special case of matrix sensing has been studied extensively through the notion of Restricted Isometry Property (RIP), leading to a wealth of results on the geometric landscape of the problem and the convergence rate of common algorithms. However, the existing results can handle the problem in the case with a general objective function subject to noisy data only when the RIP constant is close to 0. In this paper, we develop a new mathematical framework to solve the above-mentioned problem with a far less restrictive RIP constant. We prove that as long as the RIP constant of the noiseless objective is less than $1/3$, any spurious local solution of the noisy optimization problem must be close to the ground truth solution. By working through the strict saddle property, we also show that an approximate solution can be found in polynomial time. We characterize the geometry of the spurious local minima of the problem in a local region around the ground truth in the case when the RIP constant is greater than $1/3$. Compared to the existing results in the literature, this paper offers the strongest RIP bound and provides a complete theoretical analysis on the global and local optimization landscapes of general low-rank optimization problems under random corruptions from any finite-variance family.