Proximal gradient methods beyond monotony

TL;DR

This paper extends proximal gradient methods to nonconvex, nonsmooth optimization problems using an adaptive nonmonotone approach, providing convergence guarantees and demonstrating potential benefits over traditional monotone methods.

Contribution

It introduces a novel nonmonotone proximal gradient scheme with convergence analysis for nonconvex, nonsmooth problems, surpassing limitations of existing monotone strategies.

Findings

Asymptotic convergence under weak assumptions

Global worst-case convergence rates established

Numerical example shows advantages of nonmonotonicity

Abstract

We address composite optimization problems, which consist in minimizing the sum of a smooth and a merely lower semicontinuous function, without any convexity assumptions. Numerical solutions of these problems can be obtained by proximal gradient methods, which often rely on a line search procedure as globalization mechanism. We consider an adaptive nonmonotone proximal gradient scheme based on an averaged merit function and establish asymptotic convergence guarantees under weak assumptions, delivering results on par with the monotone strategy. Global worst-case rates for the iterates and a stationarity measure are also derived. Finally, a numerical example indicates the potential of nonmonotonicity and spectral approximations.