Convergence of ZH-type nonmonotone descent method for Kurdyka-{\L}ojasiewicz optimization problems

TL;DR

This paper introduces a new iterative framework for optimizing KL functions using ZH-type nonmonotone descent methods, proving convergence and convergence rates for nonconvex, nonsmooth problems, including applications to proximal and Riemannian gradient methods.

Contribution

It is the first to establish full convergence of ZH-type nonmonotone descent methods for nonconvex, nonsmooth optimization problems with KL property.

Findings

Sequences converge to critical points under the framework.

Linear convergence rate for KL exponent θ in (0, 1/2].

Sublinear convergence rate for KL exponent θ in (1/2, 1).

Abstract

We propose a novel iterative framework for minimizing a proper lower semicontinuous Kurdyka-{\L}ojasiewicz (KL) function $\Phi$. It comprises a Zhang-Hager (ZH-type) nonmonotone decrease condition and a relative error condition. Hence, the sequence generated by the ZH-type nonmonotone descent methods will fall within this framework. Any sequence conforming to this framework is proved to converge to a critical point of $\Phi$. If in addition $\Phi$ has the KL property of exponent $\theta\!\in(0,1)$ at the critical point, the convergence has a linear rate for $\theta\in(0,1/2]$ and a sublinear rate of exponent $\frac{1-\theta}{1-2\theta}$ for $\theta\in(1/2,1)$. To the best of our knowledge, this is the first work to establish the full convergence of the iterate sequence generated by a ZH-type nonmonotone descent method for nonconvex and nonsmooth optimization problems. The obtained results are also applied to achieve the full convergence of the iterate sequences produced by the proximal gradient method and Riemannian gradient method with the ZH-type nonmonotone line-search.