A Superlinear Convergence Framework for Kurdyka-{\L}ojasiewicz Optimization

TL;DR

This paper develops a framework ensuring superlinear convergence for nonconvex nonsmooth optimization, extending previous methods and applying it to cubic regularization techniques for problems with the KL property.

Contribution

It extends an iterative framework to guarantee Q-superlinear convergence for nonconvex nonsmooth optimization and applies it to inexact cubic regularization methods.

Findings

Sequences satisfying the framework are globally convergent for KL functions.

Achieves Q-superlinear convergence rate of 4/3 for inexact cubic regularization.

Framework applies to composite problems with KL exponent 1/2.

Abstract

This work extends the iterative framework proposed by Attouch et al. (in Math. Program. 137: 91-129, 2013) for minimizing a nonconvex and nonsmooth function $\Phi$ so that the generated sequence possesses a Q-superlinear convergence rate. This framework consists of a monotone decrease condition, a relative error condition and a continuity condition, and the first two conditions both involve a parameter $p\!>0$. We justify that any sequence conforming to this framework is globally convergent when $\Phi$ is a Kurdyka-{\L}ojasiewicz (KL) function, and the convergence has a Q-superlinear rate of order $\frac{p}{\theta(1+p)}$ when $\Phi$ is a KL function of exponent $\theta\in(0,\frac{p}{p+1})$. Then, we illustrate that the iterate sequence generated by an inexact $q\in[2,3]$-order regularization method for composite optimization problems with a nonconvex and nonsmooth term belongs to this framework, and consequently, first achieve the Q-superlinear convergence rate of order $4/3$ for an inexact cubic regularization method to solve this class of composite problems with KL property of exponent $1/2$.