Adaptive Consensus: A network pruning approach for decentralized optimization

TL;DR

This paper introduces an adaptive, communication-efficient framework for decentralized optimization that reduces communication costs by selectively tracking disagreement errors and choosing influential edges, with proven convergence guarantees.

Contribution

It proposes novel adaptive algorithms for decentralized consensus and optimization that significantly cut communication while maintaining convergence, supported by theoretical analysis.

Findings

Algorithms achieve reduced communication without sacrificing convergence.

Theoretical guarantees confirm the effectiveness of the adaptive approach.

Numerical experiments demonstrate substantial communication savings.

Abstract

We consider network-based decentralized optimization problems, where each node in the network possesses a local function and the objective is to collectively attain a consensus solution that minimizes the sum of all the local functions. A major challenge in decentralized optimization is the reliance on communication which remains a considerable bottleneck in many applications. To address this challenge, we propose an adaptive randomized communication-efficient algorithmic framework that reduces the volume of communication by periodically tracking the disagreement error and judiciously selecting the most influential and effective edges at each node for communication. Within this framework, we present two algorithms: Adaptive Consensus (AC) to solve the consensus problem and Adaptive Consensus based Gradient Tracking (AC-GT) to solve smooth strongly convex decentralized optimization problems. We establish strong theoretical convergence guarantees for the proposed algorithms and quantify their performance in terms of various algorithmic parameters under standard assumptions. Finally, numerical experiments showcase the effectiveness of the framework in significantly reducing the information exchange required to achieve a consensus solution.