Distributed Optimization over Time-varying Graphs with Imperfect Sharing of Information

TL;DR

This paper introduces a two time-scale decentralized gradient descent algorithm for strongly convex distributed optimization over time-varying graphs with imperfect information sharing, achieving convergence at a rate of O(T^{-1/2}).

Contribution

It proposes a novel two time-scale algorithm that handles lossy information sharing and diminishing weights in time-varying graphs for distributed optimization.

Findings

Convergence to the global optimum at rate O(T^{-1/2}) for strongly convex functions.

New tools for dealing with diminishing average weights over time-varying graphs.

Effective handling of lossy information sharing in decentralized settings.

Abstract

We study strongly convex distributed optimization problems where a set of agents are interested in solving a separable optimization problem collaboratively. In this paper, we propose and study a two time-scale decentralized gradient descent algorithm for a broad class of lossy sharing of information over time-varying graphs. One time-scale fades out the (lossy) incoming information from neighboring agents, and one time-scale regulates the local loss functions' gradients. For strongly convex loss functions, with a proper choice of step-sizes, we show that the agents' estimates converge to the global optimal state at a rate of $O(T^{-1/2})$. Another important contribution of this work is to provide novel tools to deal with diminishing average weights over time-varying graphs.