Communication-Efficient Decentralized Optimization via Double-Communication Symmetric ADMM

TL;DR

This paper introduces a novel decentralized symmetric ADMM algorithm that enhances communication efficiency by incorporating multiple communication rounds per iteration, reducing total communication cost while maintaining linear convergence.

Contribution

The paper proposes a new decentralized symmetric ADMM with multiple communication rounds, improving communication efficiency and convergence in networked optimization without a central coordinator.

Findings

Achieves linear convergence under weak assumptions.

Reduces total communication cost compared to existing methods.

Validated through experiments on regression and classification tasks.

Abstract

This paper focuses on decentralized composite optimization over networks without a central coordinator. We propose a novel decentralized symmetric ADMM algorithm that incorporates multiple communication rounds within each iteration, derived from a new constraint formulation that enables information exchange beyond immediate neighbors. While increasing per-iteration communication, our approach significantly reduces the total number of iterations and overall com- munication cost. We further design optimal communication rules that minimize the number of rounds and variables transmitted per iteration. The proposed algorithm is shown to achieve linear convergence under standard and relatively weak assumptions (e.g., metric subregularity). Extensive experiments on regression and classification tasks validate the theoretical results and demonstrate superior performance compared to existing decentralized optimization methods.