When Are Nonconvex Problems Not Scary?

TL;DR

This paper discusses conditions under which smooth nonconvex problems are globally solvable, and introduces a second-order trust-region algorithm that guarantees convergence to a global minimum without special initializations.

Contribution

The paper identifies structural properties that make certain nonconvex problems globally solvable and proposes an efficient second-order algorithm with convergence guarantees.

Findings

Algorithm converges to global minimizer efficiently

Applicable to problems like dictionary learning and phase retrieval

No need for special initializations

Abstract

In this note, we focus on smooth nonconvex optimization problems that obey: (1) all local minimizers are also global; and (2) around any saddle point or local maximizer, the objective has a negative directional curvature. Concrete applications such as dictionary learning, generalized phase retrieval, and orthogonal tensor decomposition are known to induce such structures. We describe a second-order trust-region algorithm that provably converges to a global minimizer efficiently, without special initializations. Finally we highlight alternatives, and open problems in this direction.