Non-Shannon Information Inequalities in Four Random Variables

TL;DR

This paper explores non-Shannon information inequalities in four variables, deriving new inequalities using auxiliary variables, and demonstrates their applications in network coding and entropy space analysis.

Contribution

It shows that the Zhang-Yeung inequality can be derived with one auxiliary variable and introduces many new non-Shannon inequalities in four variables.

Findings

Derived the Zhang-Yeung inequality from a single auxiliary variable.

Generated numerous new non-Shannon inequalities in four variables.

Applied inequalities to bound information flow in network coding.

Abstract

Any unconstrained information inequality in three or fewer random variables can be written as a linear combination of instances of Shannon's inequality I(A;B|C) >= 0 . Such inequalities are sometimes referred to as "Shannon" inequalities. In 1998, Zhang and Yeung gave the first example of a "non-Shannon" information inequality in four variables. Their technique was to add two auxiliary variables with special properties and then apply Shannon inequalities to the enlarged list. Here we will show that the Zhang-Yeung inequality can actually be derived from just one auxiliary variable. Then we use their same basic technique of adding auxiliary variables to give many other non-Shannon inequalities in four variables. Our list includes the inequalities found by Xu, Wang, and Sun, but it is by no means exhaustive. Furthermore, some of the inequalities obtained may be superseded by stronger inequalities that have yet to be found. Indeed, we show that the Zhang-Yeung inequality is one of those that is superseded. We also present several infinite families of inequalities. This list includes some, but not all of the infinite families found by Matus. Then we will give a description of what additional information these inequalities tell us about entropy space. This will include a conjecture on the maximum possible failure of Ingleton's inequality. Finally, we will present an application of non-Shannon inequalities to network coding. We will demonstrate how these inequalities are useful in finding bounds on the information that can flow through a particular network called the Vamos network.