Decentralized Data Reduction with Quantization Constraints

TL;DR

This paper extends the sufficiency principle to decentralized inference, analyzing how quantization constraints affect data reduction and identifying conditions where sufficiency-based reduction remains optimal.

Contribution

It introduces a framework for decentralized data reduction under quantization constraints and characterizes when sufficiency-based methods are optimal.

Findings

Sufficiency-based data reduction is not always optimal under quantization.

Optimality holds when data are conditionally independent across nodes.

Conditional independence via hidden variables can restore sufficiency-based optimality.

Abstract

A guiding principle for data reduction in statistical inference is the sufficiency principle. This paper extends the classical sufficiency principle to decentralized inference, i.e., data reduction needs to be achieved in a decentralized manner. We examine the notions of local and global sufficient statistics and the relationship between the two for decentralized inference under different observation models. We then consider the impacts of quantization on decentralized data reduction which is often needed when communications among sensors are subject to finite capacity constraints. The central question we intend to ask is: if each node in a decentralized inference system has to summarize its data using a finite number of bits, is it still optimal to implement data reduction using global sufficient statistics prior to quantization? We show that the answer is negative using a simple example and proceed to identify conditions under which sufficiency based data reduction followed by quantization is indeed optimal. They include the well known case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the introduction of an appropriately chosen hidden variable.