Distributed Semi-Stochastic Optimization with Quantization Refinement

TL;DR

This paper introduces a distributed optimization algorithm for communication-constrained networks that uses semi-stochastic proximal gradient methods with iterative quantization refinement to ensure efficient convergence.

Contribution

It proposes a novel distributed optimization algorithm combining semi-stochastic methods with quantization refinement, achieving linear convergence under communication constraints.

Findings

Algorithm achieves linear convergence rate.

Quantization refinement reduces message size effectively.

Numerical simulations demonstrate strong performance.

Abstract

We consider the problem of regularized regression in a network of communication-constrained devices. Each node has local data and objectives, and the goal is for the nodes to optimize a global objective. We develop a distributed optimization algorithm that is based on recent work on semi-stochastic proximal gradient methods. Our algorithm employs iteratively refined quantization to limit message size. We present theoretical analysis and conditions for the algorithm to achieve a linear convergence rate. Finally, we demonstrate the performance of our algorithm through numerical simulations.