Convergence Rate of Inertial Forward-Backward Algorithms Based on the Local Error Bound Condition

TL;DR

This paper analyzes the convergence rates of inertial forward-backward algorithms, including FISTA variants, under local error bound conditions, establishing strong convergence and improved rates with theoretical and numerical validation.

Contribution

It introduces new assumptions on the parameter sequence in IFB algorithms, proving enhanced convergence rates and strong convergence of iterates under local error bounds.

Findings

FISTA with modified parameters achieves sublinear convergence o(1/k^p) for any p>1.

Strong convergence of iterates is established for FISTA under local error bounds.

Numerical experiments validate the theoretical convergence improvements.

Abstract

The "Inertial Forward-Backward algorithm" (IFB) is a powerful tool for convex nonsmooth minimization problems, it gives the well known "fast iterative shrinkage-thresholding algorithm " (FISTA), which enjoys $O\left( {\frac{1}{{{k^2}}}} \right)$ global convergence rate of function values, however, no convergence of iterates has been proved; by do a small modification, an accelerated IFB called "FISTA\_CD" improves the convergence rate of function values to $o\left( {\frac{1}{{{k^2}}}} \right)$ and shows the weak convergence of iterates. The local error bound condition is extremely useful in analyzing the convergence rates of a host of iterative methods for solving optimization problems, and in practical application, a large number of problems with special structure often satisfy the error bound condition. Naturally, using local error bound condition to derive or improve the convergence rate of IFB is a common means. In this paper, based on the local error bound condition, we exploit an new assumption condition for the important parameter $t_k$ in IFB, and establish the improved convergence rate of function values and strong convergence of the iterates generated by the IFB algorithms with six $t_k$ satisfying the above assumption condition in Hilbert space. It is remarkable that, under the local error bound condition, we establish the strong convergence of the iterates generated by the original FISTA, and prove that the convergence rates of function values for FISTA\_CD is actually related to the value of parameter $a,$ and show that the IFB algorithms with some $t_k$ mentioned above can achieve sublinear convergence rate $o\left( {\frac{1}{{{k^p}}}} \right)$ for any positive integer $p>1$. Some numerical experiments are conducted to illustrate our results.