Mini-batch stochastic three-operator splitting for distributed optimization

TL;DR

This paper introduces a stochastic primal-dual algorithm for distributed optimization where agents handle nonsmooth convex functions and stochastic data, ensuring convergence despite heavy-tailed noise.

Contribution

It proposes a novel stochastic extension of the triangular pre-conditioned primal-dual algorithm for distributed nonsmooth convex optimization with stochastic data.

Findings

Almost sure convergence of the proposed algorithm

Effective handling of heavy-tailed stochastic noise

Validated performance through numerical experiments

Abstract

We consider a network of agents, each with its own private cost consisting of a sum of two possibly nonsmooth convex functions, one of which is composed with a linear operator. At every iteration each agent performs local calculations and can only communicate with its neighbors. The challenging aspect of our study is that the smooth part of the private cost function is given as an expected value and agents only have access to this part of the problem formulation via a heavy-tailed stochastic oracle. To tackle such sampling-based optimization problems, we propose a stochastic extension of the triangular pre-conditioned primal-dual algorithm. We demonstrate almost sure convergence of the scheme and validate the performance of the method via numerical experiments.