A first-order stochastic primal-dual algorithm with correction step

TL;DR

This paper introduces a stochastic primal-dual splitting algorithm with correction steps for structured monotone inclusions, demonstrating its convergence properties and extending previous saddle point methods to stochastic settings.

Contribution

It proposes a novel stochastic primal-dual algorithm with correction steps for monotone inclusions, extending existing saddle point algorithms to stochastic scenarios.

Findings

Proves weak almost sure convergence of the algorithm.

Establishes ergodic convergence in expectation for saddle point models.

Shows the method's applicability to structured monotone inclusions.

Abstract

We investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in {\em Y. Drori, S. Sabach, and M. Teboulle, A simple algorithm for a class of nonsmooth convex-concave saddle-point problems, 2015} and {\em I. Loris and C. Verhoeven, On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty, 2011} for saddle point problems. It consists in a forward step determined by the stochastic evaluation of the cocoercive operator, a backward step in the dual variables involving the resolvent of the monotone operator, and an additional forward step using the stochastic evaluation of the cocoercive introduced in the first step. We prove weak almost sure convergence of the iterates by showing that the primal-dual sequence generated by the method is stochastic quasi Fej\'er-monotone with respect to the set of zeros of the considered primal and dual inclusions. Additional results on ergodic convergence in expectation are considered for the special case of saddle point models.