Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates

TL;DR

This paper presents a two-timescale primal-dual algorithm for decentralized optimization over unreliable networks, improving communication efficiency and robustness while ensuring convergence to stationary solutions.

Contribution

Introduces TiCoPD, a novel two-timescale compressed primal-dual method combining majorization-minimization and consensus techniques for unreliable networks.

Findings

Algorithm converges to stationary solutions for smooth, non-convex objectives.

Enhances communication efficiency via compression of shared differences.

Numerical experiments validate robustness and convergence properties.

Abstract

This paper introduces a robust two-timescale compressed primal-dual (TiCoPD) algorithm tailored for decentralized optimization under bandwidth-limited and unreliable channels. By integrating the majorization-minimization approach with the primal-dual optimization framework, the TiCoPD algorithm strategically compresses the difference term shared among agents to enhance communication efficiency and robustness against noisy channels without compromising convergence stability. The method incorporates a mirror sequence for agent consensus on nonlinearly compressed terms updated on a fast timescale, together with a slow timescale primal-dual recursion for optimizing the objective function. Our analysis demonstrates that the proposed algorithm converges to a stationary solution when the objective function is smooth but possibly non-convex. Numerical experiments corroborate the conclusions of this paper.