Entropic Inequalities and Marginal Problems

TL;DR

This paper explores how Shannon entropic inequalities can serve as necessary conditions for the existence of joint distributions in marginal problems, providing computational tools and analytic solutions, especially for cycle graphs.

Contribution

It introduces a framework linking entropic inequalities to marginal problems, offers a software tool for their computation, and provides a complete analytic classification for cycle graphs.

Findings

Entropic inequalities are necessary conditions for joint distributions.

Complete analytic solution for cycle graph marginal problems.

Non-Shannon inequalities are relevant for continuous variables.

Abstract

A marginal problem asks whether a given family of marginal distributions for some set of random variables arises from some joint distribution of these variables. Here we point out that the existence of such a joint distribution imposes non-trivial conditions already on the level of Shannon entropies of the given marginals. These entropic inequalities are necessary (but not sufficient) criteria for the existence of a joint distribution. For every marginal problem, a list of such Shannon-type entropic inequalities can be calculated by Fourier-Motzkin elimination, and we offer a software interface to a Fourier-Motzkin solver for doing so. For the case that the hypergraph of given marginals is a cycle graph, we provide a complete analytic solution to the problem of classifying all relevant entropic inequalities, and use this result to bound the decay of correlations in stochastic processes. Furthermore, we show that Shannon-type inequalities for differential entropies are not relevant for continuous-variable marginal problems; non-Shannon-type inequalities are, both in the discrete and in the continuous case. In contrast to other approaches, our general framework easily adapts to situations where one has additional (conditional) independence requirements on the joint distribution, as in the case of graphical models. We end with a list of open problems. A complementary article discusses applications to quantum nonlocality and contextuality.