Spatio-Temporal Communication Compression in Distributed Prime-Dual Flows

TL;DR

This paper introduces a unified framework for spatio-temporal communication compression in distributed multi-agent optimization, demonstrating exponential convergence of the proposed algorithms with various compressors.

Contribution

It proposes a general spatio-temporal compressor framework and develops new distributed prime-dual flows with proven exponential convergence.

Findings

Several compressors fit into the spatio-temporal framework.

The proposed flows achieve exponential convergence.

Numerical examples validate theoretical results.

Abstract

In this paper, we study distributed prime-dual flows for multi-agent optimization with spatio-temporal compressions. The central aim of multi-agent optimization is for a network of agents to collaboratively solve a system-level optimization problem with local objective functions and node-to-node communication by distributed algorithms. The scalability of such algorithms crucially depends on the complexity of the communication messages, and a number of communication compressors for distributed optimization have recently been proposed in the literature. First of all, we introduce a general spatio-temporal compressor characterized by the stability of the resulting dynamical system along the vector field of the compressor. We show that several important distributed optimization compressors such as the greedy sparsifier, the uniform quantizer, and the scalarizer all fall into the category of this spatio-temporal compressor. Next, we propose two distributed prime-dual flows with the spatio-temporal compressors being applied to local node states and local error states, respectively, and prove (exponential) convergence of the node trajectories to the global optimizer for (strongly) convex cost functions. Finally, a few numerical examples are present to illustrate our theoretical results.