Linear Convergence of Proximal Gradient Algorithm with Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

TL;DR

This paper proves that the proximal gradient algorithm with extrapolation converges linearly for a class of nonconvex nonsmooth problems under certain conditions, extending known results for convex cases.

Contribution

It establishes $R$-linear convergence of the proximal gradient method with extrapolation for nonconvex problems under an error bound condition, including cases with fixed restart.

Findings

Convergence is $R$-linear when extrapolation coefficients are below a threshold.

The threshold equals 1 for convex problems, ensuring linear convergence of FISTA with restart.

Numerical experiments support the theoretical convergence results.

Abstract

In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a threshold such that if the extrapolation coefficients are chosen below this threshold, then the sequence generated converges $R$-linearly to a stationary point of the problem. Moreover, the corresponding sequence of objective values is also $R$-linearly convergent. In addition, the threshold reduces to $1$ for convex problems and, as a consequence, we obtain the $R$-linear convergence of the sequence generated by FISTA with fixed restart. Finally, we present some numerical experiments to illustrate our results.