Is Consensus Acceleration Possible in Decentralized Optimization over Slowly Time-Varying Networks?

TL;DR

This paper investigates the potential for accelerating consensus in decentralized optimization over networks with limited and slow changes, providing new bounds and algorithms for such scenarios.

Contribution

It derives lower bounds on communication complexity for various network change regimes and introduces an accelerated consensus algorithm for certain slowly varying networks.

Findings

Lower bounds established for communication complexity in different network change regimes.

An accelerated consensus algorithm demonstrated for specific slowly time-varying networks.

Consensus acceleration is possible under limited network change conditions.

Abstract

We consider decentralized optimization problems where one aims to minimize a sum of convex smooth objective functions distributed between nodes in the network. The links in the network can change from time to time. For the setting when the amount of changes is arbitrary, lower complexity bounds and corresponding optimal algorithms are known, and the consensus acceleration is not possible. However, in practice the magnitude of network changes may be limited. We derive lower communication complexity bounds for several regimes of velocity of networks changes. Moreover, we show how to obtain accelerated communication rates for a certain class of time-varying graphs using a specific consensus algorithm.