Relaxed and inertial nonlinear Forward-Backward algorithm

TL;DR

This paper introduces relaxed and inertial variants of the nonlinear Forward-Backward algorithm, providing new convergence results and demonstrating that decreasing inertial parameters can enhance numerical convergence in optimization and image restoration tasks.

Contribution

The paper extends the nonlinear Forward-Backward algorithm with inertial and relaxation steps, establishing convergence guarantees and introducing decreasing inertial sequences as a novel approach.

Findings

Decreasing inertial sequences can accelerate convergence.

New convergence guarantees for inertial and relaxed variants.

Numerical experiments confirm improved performance in optimization and image restoration.

Abstract

The Nonlinear Forward-Backward (NFB) algorithm, also known as warped resolvent iterations, is a splitting method for finding zeros of sums of monotone operators. In particular cases, NFB reduces to well-known algorithms such as Forward-Backward, Forward-Backward-Forward, Chambolle--Pock, and Condat--V\~u. Therefore, NFB can be used to solve monotone inclusions involving sums of maximally monotone, cocoercive, monotone and Lipschitz operators as well as linear compositions terms. In this article, we study the weak and strong (linear) convergence of NFB with inertial and relaxation steps. Our results recover known convergence guarantees for the aforementioned methods when extended with inertial and relaxation terms. Additionally, we establish the convergence of inertial and relaxed variants of the Forward-Backward-Half-Forward and Forward-Primal--Dual-Half-Forward algorithms, which, to the best of our knowledge, are new contributions. We consider both nondecreasing and decreasing sequences of inertial parameters, the latter being a novel approach in the context of inertial algorithms. To evaluate the performance of these strategies, we present numerical experiments on optimization problems with affine constraints and on image restoration tasks. Our results show that decreasing inertial sequences can accelerate the numerical convergence of the algorithms.