Distributed Optimization by Network Flows with Spatio-Temporal Compression

TL;DR

This paper introduces the concept of spatio-temporal compressors in distributed optimization, demonstrating their effectiveness in reducing communication and ensuring exponential convergence in networked systems.

Contribution

It proposes a new class of spatio-temporal compressors, integrates them into continuous-time consensus and primal-dual flows, and develops an observer-based flow with broader convergence guarantees.

Findings

Spatio-temporal compressors effectively reduce communication in distributed algorithms.

The proposed flows achieve exponential convergence to the global optimum.

Numerical simulations confirm the versatility and convergence of the methods.

Abstract

Several data compressors have been proposed in distributed optimization frameworks of network systems to reduce communication overhead in large-scale applications. In this paper, we demonstrate that effective information compression may occur over time or space during sequences of node communications in distributed algorithms, leading to the concept of spatio-temporal compressors. This abstraction classifies existing compressors and inspires new compressors as spatio-temporal compressors, with their effectiveness described by constructive stability criteria from nonlinear system theory. Subsequently, we incorporate these spatio-temporal compressors directly into standard continuous-time consensus flows and distributed primal-dual flows, establishing conditions ensuring exponential convergence. Additionally, we introduce a novel observer-based distributed primal-dual continuous flow integrated with spatio-temporal compressors, which provides broader convergence conditions. These continuous flows achieve exponential convergence to the global optimum when the objective function is strongly convex and can be discretized using Euler approximations. Finally, numerical simulations illustrate the versatility of the proposed spatio-temporal compressors and verify the convergence of