A Stochastic Approximation Approach for Efficient Decentralized Optimization on Random Networks

TL;DR

This paper introduces a stochastic approximation framework with novel algorithms for decentralized optimization over random networks, achieving faster convergence and efficient communication under challenging network conditions.

Contribution

It develops the FSPDA framework with two algorithms supporting sparsified communication and variance reduction, improving convergence rates in decentralized optimization.

Findings

FSPDA-SA achieves $oldsymbol{O}(1/\sqrt{T})$ convergence rate.

FSPDA-STORM achieves $oldsymbol{O}(1/T^{2/3})$ convergence rate.

Numerical experiments validate the effectiveness of the proposed algorithms.

Abstract

A challenging problem in decentralized optimization is to develop algorithms with fast convergence on random and time varying topologies under unreliable and bandwidth-constrained communication network. This paper studies a stochastic approximation approach with a Fully Stochastic Primal Dual Algorithm (FSPDA) framework. Our framework relies on a novel observation that randomness in time varying topology can be incorporated in a stochastic augmented Lagrangian formulation, whose expected value admits saddle points that coincide with stationary solutions of the decentralized optimization problem. With the FSPDA framework, we develop two new algorithms supporting efficient sparsified communication on random time varying topologies -- FSPDA-SA allows agents to execute multiple local gradient steps depending on the time varying topology to accelerate convergence, and FSPDA-STORM further incorporates a variance reduction step to improve sample complexity. For problems with smooth (possibly non-convex) objective function, within $T$ iterations, we show that FSPDA-SA (resp. FSPDA-STORM) finds an $\mathcal{O}( 1/\sqrt{T} )$-stationary (resp. $\mathcal{O}( 1/T^{2/3} )$) solution. Numerical experiments show the benefits of the FSPDA algorithms.