Worst-case Compressibility of Discrete and Finite Distributions

TL;DR

This paper investigates the worst-case compressibility of discrete distributions in distributed source coding, introducing new measures and analyzing how support-set properties influence data gathering efficiency.

Contribution

It introduces the concepts of bit- and informant-compressibility of support-sets and analyzes their regions in distributed source coding and function computation.

Findings

Support-set cardinality does not necessarily reduce informant bits needed.

Bit- and informant-compressibility regions depend on support-set properties.

Formal measures for worst-case data compression are developed.

Abstract

In the worst-case distributed source coding (DSC) problem of [1], the smaller cardinality of the support-set describing the correlation in informant data, may neither imply that fewer informant bits are required nor that fewer informants need to be queried, to finish the data-gathering at the sink. It is important to formally address these observations for two reasons: first, to develop good worst-case information measures and second, to perform meaningful worst-case information-theoretic analysis of various distributed data-gathering problems. Towards this goal, we introduce the notions of bit-compressibility and informant-compressibility of support-sets. We consider DSC and distributed function computation problems and provide results on computing the bit- and informant- compressibilities regions of the support-sets as a function of their cardinality.