Global Convergence of Model Function Based Bregman Proximal Minimization Algorithms

TL;DR

This paper introduces the MAP property to extend convergence guarantees to a broad class of nonconvex nonsmooth optimization problems, and proposes the Model BPG algorithm with demonstrated superior performance.

Contribution

It proposes the MAP property for nonconvex nonsmooth problems and develops the Model BPG algorithm with global convergence analysis.

Findings

Model BPG outperforms existing methods on phase retrieval and inverse problems.

The convergence is established via a new Lyapunov function.

Numerical results show superior performance of Model BPG.

Abstract

Lipschitz continuity of the gradient mapping of a continuously differentiable function plays a crucial role in designing various optimization algorithms. However, many functions arising in practical applications such as low rank matrix factorization or deep neural network problems do not have a Lipschitz continuous gradient. This led to the development of a generalized notion known as the $L$-smad property, which is based on generalized proximity measures called Bregman distances. However, the $L$-smad property cannot handle nonsmooth functions, for example, simple nonsmooth functions like $\abs{x^4-1}$ and also many practical composite problems are out of scope. We fix this issue by proposing the MAP property, which generalizes the $L$-smad property and is also valid for a large class of nonconvex nonsmooth composite problems. Based on the proposed MAP property, we propose a globally convergent algorithm called Model BPG, that unifies several existing algorithms. The convergence analysis is based on a new Lyapunov function. We also numerically illustrate the superior performance of Model BPG on standard phase retrieval problems, robust phase retrieval problems, and Poisson linear inverse problems, when compared to a state of the art optimization method that is valid for generic nonconvex nonsmooth optimization problems.