Forward--Backward Splitting with Deviations for Monotone Inclusions

TL;DR

This paper introduces a flexible, deviation-augmented forward-backward algorithm for monotone inclusions, ensuring convergence and enabling improved performance in related algorithms through a novel deviation-based approach.

Contribution

It presents a weakly convergent forward-backward variant with deviation vectors, allowing for flexible algorithm design and convergence guarantees, including linear convergence under certain conditions.

Findings

Algorithm guarantees convergence with bounded deviations

Deviations can improve algorithm performance

Numerical experiments confirm theoretical results

Abstract

We propose and study a weakly convergent variant of the forward--backward algorithm for solving structured monotone inclusion problems. Our algorithm features a per-iteration deviation vector which provides additional degrees of freedom. The only requirement on the deviation vector to guarantee convergence is that its norm is bounded by a quantity that can be computed online. This approach provides great flexibility and opens up for the design of new and improved forward--backward-based algorithms, while retaining global convergence guarantees. These guarantees include linear convergence of our method under a metric subregularity assumption without the need to adapt the algorithm parameters. Choosing suitable monotone operators allows for incorporating deviations into other algorithms, such as Chambolle--Pock and Krasnoselsky--Mann iterations. We propose a novel inertial primal--dual algorithm by selecting the deviations along a momentum direction and deciding their size using the norm condition. Numerical experiments demonstrate our convergence claims and show that even this simple choice of deviation vector can improve the performance, compared, e.g., to the standard Chambolle--Pock algorithm.