Incorporating History and Deviations in Forward--Backward Splitting

TL;DR

This paper introduces a flexible variation of the forward--backward splitting method that incorporates past iterates and deviation vectors, enabling interpolation between existing schemes and achieving faster convergence in minimax problems.

Contribution

It proposes a novel algorithm that integrates deviation vectors and a scalar parameter to interpolate between standard and accelerated forward--backward methods, improving convergence.

Findings

The new method outperforms existing schemes on minimax problems.

Special cases include accelerated proximal point and Halpern iteration.

The approach allows flexible deviation vector choices satisfying a norm condition.

Abstract

We propose a variation of the forward--backward splitting method for solving structured monotone inclusions. Our method integrates past iterates and two deviation vectors into the update equations. These deviation vectors bring flexibility to the algorithm and can be chosen arbitrarily as long as they together satisfy a norm condition. We present special cases where the deviation vectors, selected as predetermined linear combinations of previous iterates, always meet the norm condition. Notably, we introduce an algorithm employing a scalar parameter to interpolate between the conventional forward--backward splitting scheme and an accelerated O(1/n^2)-convergent forward--backward method that encompasses both the accelerated proximal point method and the Halpern iteration as special cases. The existing methods correspond to the two extremes of the allowed scalar parameter range. By choosing the interpolation scalar near the midpoint of the permissible range, our algorithm significantly outperforms these previously known methods when addressing a basic monotone inclusion problem stemming from minimax optimization.