Correlated quantization for distributed mean estimation and optimization

TL;DR

This paper introduces a correlated quantization method for distributed mean estimation that adapts to data deviation, improving convergence and outperforming existing protocols without prior data knowledge.

Contribution

The paper presents a novel correlated quantization protocol that depends on data deviation, not range, and enhances distributed optimization convergence without prior data assumptions.

Findings

Outperforms existing mean estimation protocols in experiments

Achieves better convergence rates in distributed optimization

Proven to be optimal under mild assumptions

Abstract

We study the problem of distributed mean estimation and optimization under communication constraints. We propose a correlated quantization protocol whose leading term in the error guarantee depends on the mean deviation of data points rather than only their absolute range. The design doesn't need any prior knowledge on the concentration property of the dataset, which is required to get such dependence in previous works. We show that applying the proposed protocol as sub-routine in distributed optimization algorithms leads to better convergence rates. We also prove the optimality of our protocol under mild assumptions. Experimental results show that our proposed algorithm outperforms existing mean estimation protocols on a diverse set of tasks.