Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization
Angelia Nedi\'c, Alex Olshevsky, Michael G. Rabbat

TL;DR
This paper surveys recent advances in decentralized optimization, emphasizing how network topology influences algorithm performance and convergence rates across various communication structures.
Contribution
It provides a comprehensive overview of state-of-the-art algorithms and analyses, highlighting the impact of network structure on optimization efficiency.
Findings
Convergence rates depend on network topology and communication patterns.
Directed and time-varying networks pose unique challenges.
Topology significantly affects algorithm robustness and speed.
Abstract
In decentralized optimization, nodes cooperate to minimize an overall objective function that is the sum (or average) of per-node private objective functions. Algorithms interleave local computations with communication among all or a subset of the nodes. Motivated by a variety of applications---distributed estimation in sensor networks, fitting models to massive data sets, and distributed control of multi-robot systems, to name a few---significant advances have been made towards the development of robust, practical algorithms with theoretical performance guarantees. This paper presents an overview of recent work in this area. In general, rates of convergence depend not only on the number of nodes involved and the desired level of accuracy, but also on the structure and nature of the network over which nodes communicate (e.g., whether links are directed or undirected, static or…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
