A Stochastic forward-backward splitting method for solving monotone inclusions in Hilbert spaces

TL;DR

This paper introduces a stochastic forward-backward splitting algorithm for monotone inclusions in Hilbert spaces, providing convergence analysis and practical advantages for variational inequalities and convex optimization.

Contribution

It presents a novel stochastic extension of the classical forward-backward method with convergence guarantees and avoids averaging, beneficial for sparse and accelerated optimization.

Findings

Non-asymptotic error bounds in expectation for strongly monotone cases

Almost sure convergence under weaker assumptions

Matching convergence rates with accelerated stochastic methods

Abstract

We propose and analyze the convergence of a novel stochastic forward-backward splitting algorithm for solving monotone inclusions given by the sum of a maximal monotone operator and a single-valued maximal monotone cocoercive operator. This latter framework has a number of interesting special cases, including variational inequalities and convex minimization problems, while stochastic approaches are practically relevant to account for perturbations in the data. The algorithm we propose is a stochastic extension of the classical deterministic forward-backward method, and is obtained considering the composition of the resolvent of the maximal monotone operator with a forward step based on a stochastic estimate of the single-valued operator. Our study provides a non-asymptotic error analysis in expectation for the strongly monotone case, as well as almost sure convergence under weaker assumptions. The approach we consider allows to avoid averaging, a feature critical when considering methods based on sparsity, and, for minimization problems, it allows to obtain convergence rates matching those obtained by stochastic extensions of so called accelerated methods. Stochastic quasi Fejer's sequences are a key technical tool to prove almost sure convergence.