Relaxed and Inertial Nonlinear Forward-Backward with Momentum

TL;DR

This paper introduces inertial and relaxed variants of the nonlinear forward-backward with momentum algorithm for solving monotone inclusions, demonstrating improved convergence and performance in image restoration tasks.

Contribution

It proposes new inertial and relaxed algorithms extending NFBM, with theoretical convergence analysis and practical improvements demonstrated through numerical experiments.

Findings

Inertial and relaxed NFBM algorithms converge under certain conditions.

Numerical experiments show improved convergence with inertial and relaxation strategies.

Restart strategies further enhance convergence speed and stability.

Abstract

In this article, we study inertial algorithms for numerically solving monotone inclusions involving the sum of a maximally monotone and a cocoercive operator. In particular, we analyze the convergence of inertial and relaxed versions of the nonlinear forward-backward with momentum (NFBM). We propose an inertial version including a relaxation step, and a second version considering a double-inertial step with additional momentum. By applying NFBM to specific monotone inclusions, we derive inertial and relaxed versions of algorithms such as forward-backward, forward-half-reflect-backward (FHRB), Chambolle-Pock, Condat-V\~u, among others, thereby recovering and extending previous results from the literature for solving monotone inclusions involving maximally monotone, cocoercive, monotone and Lipschitz, and linear bounded operators. We also present numerical experiments on image restoration, comparing the proposed inertial and relaxation algorithms. In particular, we compare the inertial FHRB with its non-inertial and momentum versions. Additionally, we compare the numerical convergence for larger step-sizes versus relaxation parameters and introduce a restart strategy that incorporates larger step-sizes and inertial steps to further enhance numerical convergence.