Expander Graph and Communication-Efficient Decentralized Optimization

TL;DR

This paper explores how expander graphs can be used to design communication-efficient decentralized optimization algorithms, demonstrating their near-optimality and superior performance over other regular graphs.

Contribution

It introduces methods for constructing expander graphs tailored for decentralized optimization, highlighting their advantages in reducing communication complexity.

Findings

Expander graphs significantly improve optimization performance.

Proposed constructions adapt to various node counts and degrees.

Numerical results confirm expander graphs outperform other regular graphs.

Abstract

In this paper, we discuss how to design the graph topology to reduce the communication complexity of certain algorithms for decentralized optimization. Our goal is to minimize the total communication needed to achieve a prescribed accuracy. We discover that the so-called expander graphs are near-optimal choices. We propose three approaches to construct expander graphs for different numbers of nodes and node degrees. Our numerical results show that the performance of decentralized optimization is significantly better on expander graphs than other regular graphs.