Proximal Limited-Memory Quasi-Newton Methods for Nonsmooth Nonconvex Optimization

TL;DR

This paper presents a novel proximal limited-memory quasi-Newton method for nonsmooth, nonconvex optimization, with proven convergence and improved computational efficiency demonstrated through numerical experiments.

Contribution

It introduces a new quasi-Newton scheme that handles nonsmooth, nonconvex problems with global convergence guarantees and efficient subproblem solutions.

Findings

Proven global convergence under mild assumptions.

Established convergence rates under Kurdyka--Lojasiewicz property.

Numerical results show significant speed-up over existing methods.

Abstract

We introduce a proximal limited--memory quasi--Newton scheme for minimizing the sum of a continuously differentiable function and a proper, lower semicontinuous and prox-bounded, possibly nonsmooth, function. Both functions might be nonconvex. The method builds upon the computation of scaled proximal operators and is globalized by adaptively updating a regularization parameter based on a criterion of sufficient decrease. We prove global convergence under mild assumptions and then establish convergence of the entire sequence (with rates) under the Kurdyka--Lojasiewicz property. To efficiently solve the subproblems, we exploit the compact representation of limited-memory quasi-Newton updates. We derive also a compact representation of the limited--memory Kleinmichel formula, a rank-one quasi-Newton scheme that preserves positive definiteness under the same condition as the BFGS update. Numerical results show a significant speed up compared to other methods.