Stochastic forward-backward and primal-dual approximation algorithms with application to online image restoration

TL;DR

This paper introduces stochastic variants of forward-backward and primal-dual algorithms for convex optimization, proving their convergence and demonstrating their application to online image restoration tasks.

Contribution

It develops new stochastic algorithms for convex optimization that handle non-smooth functions and stochastic errors, with proven convergence.

Findings

Algorithms converge almost surely under mild assumptions.

Application to online image restoration shows practical effectiveness.

Framework accommodates stochastic gradient and proximity evaluations.

Abstract

Stochastic approximation techniques have been used in various contexts in data science. We propose a stochastic version of the forward-backward algorithm for minimizing the sum of two convex functions, one of which is not necessarily smooth. Our framework can handle stochastic approximations of the gradient of the smooth function and allows for stochastic errors in the evaluation of the proximity operator of the nonsmooth function. The almost sure convergence of the iterates generated by the algorithm to a minimizer is established under relatively mild assumptions. We also propose a stochastic version of a popular primal-dual proximal splitting algorithm, establish its convergence, and apply it to an online image restoration problem.