Information Inequalities for Five Random Variables

TL;DR

This paper advances the understanding of the five-variable entropic region by deriving new non-Shannon inequalities using a novel maximum entropy approach, reducing computational complexity, and proposing infinite collections of inequalities.

Contribution

It introduces a new maximum entropy method for deriving five-variable non-Shannon inequalities and develops an algorithm to enumerate all extremal inequalities, significantly expanding the known entropy region.

Findings

Derived all five-variable non-Shannon inequalities for the first nine generations.

Defined two infinite collections of entropy inequalities and proved their validity.

Proposed an algorithm to enumerate all extremal inequalities in the entropic region.

Abstract

The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown. We utilize a variant of the Maximum Entropy Method to obtain five-variable non_shannon entropy inequalities, which delimit the five-variable entropy region. This method adds copies of some of the random variables in generations. A significant reduction in computational complexity, achieved through theoretical considerations and by harnessing the inherent symmetries, allowed us to calculate all five-variable non-Shannon inequalities provided by the first nine generations. Based on the results, we define two infinite collections of such inequalities and prove them to be entropy inequalities. We investigate downward-closed subsets of non-negative lattice points that parameterize these collections, and based on this, we develop an algorithm to enumerate all extremal inequalities. The discovered set of entropy inequalities is conjectured to characterize the applied method completely.