The Sufficiency Principle for Decentralized Data Reduction

TL;DR

This paper extends the sufficiency principle to decentralized inference systems, addressing both parallel and tandem networks with dependent observations, and introduces new concepts like conditional sufficiency.

Contribution

It develops a generalized sufficiency principle for decentralized data reduction, including methods for dependent observations and connections to distributed source coding.

Findings

Locally sufficient statistics are globally sufficient in parallel networks with hidden variables.

Conditional sufficiency is introduced for tandem networks.

Links between sufficiency and distributed source coding are established.

Abstract

This paper develops the sufficiency principle suitable for data reduction in decentralized inference systems. Both parallel and tandem networks are studied and we focus on the cases where observations at decentralized nodes are conditionally dependent. For a parallel network, through the introduction of a hidden variable that induces conditional independence among the observations, the locally sufficient statistics, defined with respect to the hidden variable, are shown to be globally sufficient for the parameter of inference interest. For a tandem network, the notion of conditional sufficiency is introduced and the related theories and tools are developed. Finally, connections between the sufficiency principle and some distributed source coding problems are explored.