Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization

TL;DR

This paper surveys recent advances in decentralized optimization, emphasizing how network topology influences algorithm performance and convergence rates across various communication structures.

Contribution

It provides a comprehensive overview of state-of-the-art algorithms and analyses, highlighting the impact of network structure on optimization efficiency.

Findings

Convergence rates depend on network topology and communication patterns.

Directed and time-varying networks pose unique challenges.

Topology significantly affects algorithm robustness and speed.

Abstract

In decentralized optimization, nodes cooperate to minimize an overall objective function that is the sum (or average) of per-node private objective functions. Algorithms interleave local computations with communication among all or a subset of the nodes. Motivated by a variety of applications---distributed estimation in sensor networks, fitting models to massive data sets, and distributed control of multi-robot systems, to name a few---significant advances have been made towards the development of robust, practical algorithms with theoretical performance guarantees. This paper presents an overview of recent work in this area. In general, rates of convergence depend not only on the number of nodes involved and the desired level of accuracy, but also on the structure and nature of the network over which nodes communicate (e.g., whether links are directed or undirected, static or time-varying). We survey the state-of-the-art algorithms and their analyses tailored to these different scenarios, highlighting the role of the network topology.