Distributed Optimization with Finite Bit Adaptive Quantization for Efficient Communication and Precision Enhancement

TL;DR

This paper introduces a distributed optimization algorithm that uses 3-bit adaptive quantization to efficiently communicate and accurately converge to the optimal solution with linear rate under bandwidth constraints.

Contribution

We propose a novel adaptive quantization method with zoom-in and zoom-out operations for distributed optimization, ensuring convergence under strict bandwidth limitations.

Findings

Converges to the exact optimal solution.

Achieves linear convergence rate.

Effective under 3-bit communication constraints.

Abstract

In realistic distributed optimization scenarios, individual nodes possess only partial information and communicate over bandwidth constrained channels. For this reason, the development of efficient distributed algorithms is essential. In our paper we addresses the challenge of unconstrained distributed optimization. In our scenario each node's local function exhibits strong convexity with Lipschitz continuous gradients. The exchange of information between nodes occurs through $3$-bit bandwidth-limited channels (i.e., nodes exchange messages represented by a only $3$-bits). Our proposed algorithm respects the network's bandwidth constraints by leveraging zoom-in and zoom-out operations to adjust quantizer parameters dynamically. We show that during our algorithm's operation nodes are able to converge to the exact optimal solution. Furthermore, we show that our algorithm achieves a linear convergence rate to the optimal solution. We conclude the paper with simulations that highlight our algorithm's unique characteristics.