Distributed Lossy Averaging

TL;DR

This paper analyzes the fundamental limits of distributed averaging in networks with Gaussian sources, establishing bounds on the rate distortion function and comparing distributed and centralized protocols.

Contribution

It introduces an information theoretic framework for distributed averaging, deriving bounds on the network rate distortion function for various protocols and network configurations.

Findings

Distributed protocols incur a logarithmic factor increase in rate compared to centralized protocols.

A cutset lower bound on the rate distortion function is established and nearly achieved by a centralized protocol.

Bounds for gossip-based protocols show only a double logarithmic factor increase over the lower bound.

Abstract

An information theoretic formulation of the distributed averaging problem previously studied in computer science and control is presented. We assume a network with m nodes each observing a WGN source. The nodes communicate and perform local processing with the goal of computing the average of the sources to within a prescribed mean squared error distortion. The network rate distortion function R^*(D) for a 2-node network with correlated Gaussian sources is established. A general cutset lower bound on R^*(D) is established and shown to be achievable to within a factor of 2 via a centralized protocol over a star network. A lower bound on the network rate distortion function for distributed weighted-sum protocols, which is larger in order than the cutset bound by a factor of log m is established. An upper bound on the network rate distortion function for gossip-base weighted-sum protocols, which is only log log m larger in order than the lower bound for a complete graph network, is established. The results suggest that using distributed protocols results in a factor of log m increase in order relative to centralized protocols.