Distributed Convex Optimization With Limited Communications

TL;DR

This paper introduces the DCDA algorithm for distributed convex optimization that minimizes communication by restricting message size per coordinate, with proven convergence rates and robustness to noise and quantization.

Contribution

The paper proposes a novel distributed coordinate dual averaging algorithm that reduces communication load while maintaining convergence guarantees in various network settings.

Findings

Bounded convergence rate under different communication protocols.

Robustness to noisy and quantized messages.

Effective in large-scale distributed optimization scenarios.

Abstract

In this paper, a distributed convex optimization algorithm, termed \emph{distributed coordinate dual averaging} (DCDA) algorithm, is proposed. The DCDA algorithm addresses the scenario of a large distributed optimization problem with limited communication among nodes in the network. Currently known distributed subgradient methods, such as the distributed dual averaging or the distributed alternating direction method of multipliers algorithms, assume that nodes can exchange messages of large cardinality. Such network communication capabilities are not valid in many scenarios of practical relevance. In the DCDA algorithm, on the other hand, communication of each coordinate of the optimization variable is restricted over time. For the proposed algorithm, we bound the rate of convergence under different communication protocols and network architectures. We also consider the extensions to the case of imperfect gradient knowledge and the case in which transmitted messages are corrupted by additive noise or are quantized. Relevant numerical simulations are also provided.