Nesterov Accelerated Distributed Optimization with Efficient Quantized Communication

TL;DR

This paper introduces QANM, a distributed optimization algorithm combining Nesterov acceleration and quantized consensus to efficiently solve large-scale problems with limited communication, achieving linear convergence.

Contribution

The paper proposes a novel distributed optimization method that addresses zigzag phenomena and communication constraints simultaneously, with proven convergence and practical validation.

Findings

QANM converges linearly to a neighborhood of the optimal solution.

Simulations demonstrate accelerated convergence over non-momentum methods.

The approach effectively handles quantized communication in distributed systems.

Abstract

In modern large-scale networked systems, rapidly solving optimization problems while utilizing communication resources efficiently is critical for addressing complex tasks. In this paper, we consider an unconstrained distributed optimization problem in which information exchange among nodes is governed by a directed communication graph. In our setup we focus on two key challenges. The first is the zigzag phenomenon caused by the objective functions of individual nodes having significantly different curvature along different directions. The second is that the communication channels among nodes are subject to limited bandwidth, which motivates the use of compressed (quantized) messages. To address both challenges simultaneously, we propose QANM, a distributed optimization algorithm that combines Nesterov-accelerated gradient descent with a distributed finite-time quantized consensus protocol, enabling accelerated convergence. Under strong convexity and smoothness assumptions, we show that our proposed algorithm converges linearly to a neighborhood of the optimal solution. Finally, we validate our algorithm on a distributed sensor fusion application for multi-dimensional target parameter estimation, where simulations across two distinct scenarios confirm the convergence guarantees and demonstrate clear acceleration benefits over non-momentum baselines.