Rate Distortion for Lossy In-network Function Computation: Information Dissipation and Sequential Reverse Water-Filling

TL;DR

This paper investigates the fundamental limits of distributed lossy linear function computation in tree networks, introducing a novel incremental distortion approach that improves upon classical bounds and reveals a reverse water-filling structure.

Contribution

It introduces an incremental distortion-based framework for rate bounds in distributed computation, surpassing classical cut-set bounds and revealing a reverse water-filling structure.

Findings

Outer bounds on rate distortion are tighter than classical bounds.

Inner bounds differ from outer bounds by O(√D), where D is the overall distortion.

Rate allocation follows a reverse water-filling pattern similar to Gaussian sources.

Abstract

We consider the problem of distributed lossy linear function computation in a tree network. We examine two cases: (i) data aggregation (only one sink node computes) and (ii) consensus (all nodes compute the same function). By quantifying the accumulation of information loss in distributed computing, we obtain fundamental limits on network computation rate as a function of incremental distortions (and hence incremental loss of information) along the edges of the network. The above characterization, based on quantifying distortion accumulation, offers an improvement over classical cut-set type techniques which are based on overall distortions instead of incremental distortions. This quantification of information loss qualitatively resembles information dissipation in cascaded channels [1]. Surprisingly, this accumulation effect of distortion happens even at infinite blocklength. Combining this observation with an inequality on the dominance of mean-square quantities over relative-entropy quantities, we obtain outer bounds on the rate distortion function that are tighter than classical cut-set bounds by a difference which can be arbitrarily large in both data aggregation and consensus. We also obtain inner bounds on the optimal rate using random Gaussian coding, which differ from the outer bounds by $\mathcal{O}(\sqrt{D})$, where $D$ is the overall distortion. The obtained inner and outer bounds can provide insights on rate (bit) allocations for both the data aggregation problem and the consensus problem. We show that for tree networks, the rate allocation results have a mathematical structure similar to classical reverse water-filling for parallel Gaussian sources.