Mapping the Region of Entropic Vectors with Support Enumeration & Information Geometry

TL;DR

This paper explores the complex structure of the entropy region for four or more variables by combining algebraic support enumeration and information geometry to better understand and map the boundary of entropic vectors.

Contribution

It introduces novel algebraic and geometric methods to identify supports of probability distributions that define the boundary of the entropy region for multiple variables.

Findings

Supports in the entropy region can be efficiently identified using algebraic methods.

Information geometry helps characterize the structure of distributions on these supports.

Inner bounds to the entropy region are improved through these combined techniques.

Abstract

The region of entropic vectors is a convex cone that has been shown to be at the core of many fundamental limits for problems in multiterminal data compression, network coding, and multimedia transmission. This cone has been shown to be non-polyhedral for four or more random variables, however its boundary remains unknown for four or more discrete random variables. Methods for specifying probability distributions that are in faces and on the boundary of the convex cone are derived, then utilized to map optimized inner bounds to the unknown part of the entropy region. The first method utilizes tools and algorithms from abstract algebra to efficiently determine those supports for the joint probability mass functions for four or more random variables that can, for some appropriate set of non-zero probabilities, yield entropic vectors in the gap between the best known inner and outer bounds. These supports are utilized, together with numerical optimization over non-zero probabilities, to provide inner bounds to the unknown part of the entropy region. Next, information geometry is utilized to parameterize and study the structure of probability distributions on these supports yielding entropic vectors in the faces of entropy and in the unknown part of the entropy region.