Real Acceleration of Communication Process in Distributed Algorithms with Compression

TL;DR

This paper investigates the real-world impact of data compression on communication speed in distributed optimization algorithms, considering factors beyond message size, and proposes adaptive compression strategies for improved efficiency.

Contribution

It introduces a realistic model of communication time affected by compression and develops an adaptive method to optimize compression levels in distributed algorithms.

Findings

Compression can significantly accelerate communication in practice.

Optimal compression depends on message startup time and other factors.

Adaptive compression strategies outperform fixed approaches.

Abstract

Modern applied optimization problems become more and more complex every day. Due to this fact, distributed algorithms that can speed up the process of solving an optimization problem through parallelization are of great importance. The main bottleneck of distributed algorithms is communications, which can slow down the method dramatically. One way to solve this issue is to use compression of transmitted information. In the current literature on theoretical distributed optimization, it is generally accepted that as much as we compress information, so much we reduce communication time. But in reality, the communication time depends not only on the size of the transmitted information, but also, for example, on the message startup time. In this paper, we study distributed optimization algorithms under the assumption of a more complex and closer-to-reality dependence of transmission time on compression. In particular, we describe the real speedup achieved by compression, analyze how much it makes sense to compress information, and present an adaptive way to select the power of compression depending on unknown or changing parameters of the communication process.