An information theoretic necessary condition for perfect reconstruction

TL;DR

This paper introduces a new information theoretic necessary condition for perfect reconstruction of a discrete random variable from functions, based on Shannon's lattice theory and entropic metrics, with applications to various reconstruction problems.

Contribution

It provides a synthetic, detailed description of Shannon's lattice theory and introduces a geometric interpretation leading to a necessary condition for reconstruction.

Findings

A new geometric interpretation of Shannon's lattice theory.

A necessary (and sometimes sufficient) condition for perfect reconstruction.

Illustrations with five specific examples of reconstruction problems.

Abstract

A new information theoretic condition is presented for reconstructing a discrete random variable $X$ based on the knowledge of a set of discrete functions of $X$. The reconstruction condition is derived from Shannon's 1953 lattice theory with two entropic metrics of Shannon and Rajski. Because such a theoretical material is relatively unknown and appears quite dispersed in different references, we first provide a synthetic description (with complete proofs) of its concepts, such as total, common and complementary informations. Definitions and properties of the two entropic metrics are also fully detailed and shown compatible with the lattice structure. A new geometric interpretation of such a lattice structure is then investigated that leads to a necessary (and sometimes sufficient) condition for reconstructing the discrete random variable $X$ given a set $\{ X_1,\ldots,X_{n} \}$ of elements in the lattice generated by $X$. Finally, this condition is illustrated in five specific examples of perfect reconstruction problems: reconstruction of a symmetric random variable from the knowledge of its sign and absolute value, reconstruction of a word from a set of linear combinations, reconstruction of an integer from its prime signature (fundamental theorem of arithmetic) and from its remainders modulo a set of coprime integers (Chinese remainder theorem), and reconstruction of the sorting permutation of a list from a minimal set of pairwise comparisons.