On the Entropy Region of Gaussian Random Variables

TL;DR

This paper characterizes the entropy region of three Gaussian random variables, relates it to scalar cases, and proposes minimal conditions for higher dimensions, advancing understanding of Gaussian entropy regions.

Contribution

It provides a full characterization of the entropy region for three Gaussian variables and introduces minimal necessary conditions for higher dimensions, improving previous nonminimal results.

Findings

Entropy region of three Gaussian variables is fully characterized.

Minimal conditions for n Gaussian variables are established.

New formulas and proofs related to hyperdeterminants are presented.

Abstract

Given n (discrete or continuous) random variables X_i, the (2^n-1)-dimensional vector obtained by evaluating the joint entropy of all non-empty subsets of {X_1,...,X_n} is called an entropic vector. Determining the region of entropic vectors is an important open problem with many applications in information theory. Recently, it has been shown that the entropy regions for discrete and continuous random variables, though different, can be determined from one another. An important class of continuous random variables are those that are vector-valued and jointly Gaussian. In this paper we give a full characterization of the convex cone of the entropy region of three jointly Gaussian vector-valued random variables and prove that it is the same as the convex cone of three scalar-valued Gaussian random variables and further that it yields the entire entropy region of 3 arbitrary random variables. We further determine the actual entropy region of 3 vector-valued jointly Gaussian random variables through a conjecture. For n>=4 number of random variables, we point out a set of 2^n-1-n(n+1)/2 minimal necessary and sufficient conditions that 2^n-1 numbers must satisfy in order to correspond to the entropy vector of n scalar jointly Gaussian random variables. This improves on a result of Holtz and Sturmfels which gave a nonminimal set of conditions. These constraints are related to Cayley's hyperdeterminant and hence with an eye towards characterizing the entropy region of jointly Gaussian random variables, we also present some new results in this area. We obtain a new (determinant) formula for the 2*2*2 hyperdeterminant and we also give a new (transparent) proof of the fact that the principal minors of an n*n symmetric matrix satisfy the 2*2*...*2 (up to n times) hyperdeterminant relations.