Decentralized Stochastic Proximal Gradient Descent with Variance Reduction over Time-varying Networks

TL;DR

This paper introduces DPSVRG, a decentralized stochastic proximal gradient method with variance reduction, significantly improving convergence speed over traditional DSPG in decentralized learning scenarios.

Contribution

The paper proposes DPSVRG, a novel decentralized algorithm that employs variance reduction to accelerate convergence in decentralized learning with non-smooth regularization.

Findings

DPSVRG achieves an $O(1/T)$ convergence rate for convex objectives.

DPSVRG converges faster than DSPG, with smoother loss reduction.

Experimental results confirm improved convergence across various network topologies.

Abstract

In decentralized learning, a network of nodes cooperate to minimize an overall objective function that is usually the finite-sum of their local objectives, and incorporates a non-smooth regularization term for the better generalization ability. Decentralized stochastic proximal gradient (DSPG) method is commonly used to train this type of learning models, while the convergence rate is retarded by the variance of stochastic gradients. In this paper, we propose a novel algorithm, namely DPSVRG, to accelerate the decentralized training by leveraging the variance reduction technique. The basic idea is to introduce an estimator in each node, which tracks the local full gradient periodically, to correct the stochastic gradient at each iteration. By transforming our decentralized algorithm into a centralized inexact proximal gradient algorithm with variance reduction, and controlling the bounds of error sequences, we prove that DPSVRG converges at the rate of $O(1/T)$ for general convex objectives plus a non-smooth term with $T$ as the number of iterations, while DSPG converges at the rate $O(\frac{1}{\sqrt{T}})$. Our experiments on different applications, network topologies and learning models demonstrate that DPSVRG converges much faster than DSPG, and the loss function of DPSVRG decreases smoothly along with the training epochs.