Nonlinear Forward-Backward Splitting with Momentum Correction

TL;DR

This paper introduces a nonlinear momentum correction technique for forward-backward splitting algorithms, enhancing convergence and reducing per-iteration cost across various monotone inclusion methods.

Contribution

It proposes a novel nonlinear momentum approach that simplifies convergence guarantees and unifies many existing algorithms under a common framework.

Findings

The method improves convergence properties of nonlinear resolvent algorithms.

It encompasses and extends several known primal-dual and reflected-backward methods.

A new primal-dual algorithm with an additional resolvent step is developed.

Abstract

The nonlinear, or warped, resolvent recently explored by Giselsson and B\`ui-Combettes has been used to model a large set of existing and new monotone inclusion algorithms. To establish convergent algorithms based on these resolvents, corrective projection steps are utilized in both works. We present a different way of ensuring convergence by means of a nonlinear momentum term, which in many cases leads to cheaper per-iteration cost. The expressiveness of our method is demonstrated by deriving a wide range of special cases. These cases cover and expand on the forward-reflected-backward method of Malitsky-Tam, the primal-dual methods of V\~u-Condat and Chambolle-Pock, and the forward-reflected-Douglas-Rachford method of Ryu-V\~u. A new primal-dual method that uses an extra resolvent step is also presented as well as a general approach for adding momentum to any special case of our nonlinear forward-backward method, in particular all the algorithms listed above.