Gray--Wyner and Mutual Information Regions for Doubly Symmetric Binary Sources and Gaussian Sources

TL;DR

This paper introduces a new method to solve nonconvex optimization problems in information theory, providing explicit expressions for the Gray--Wyner and mutual information regions for DSBS and Gaussian sources, confirming longstanding conjectures.

Contribution

It offers the first analytic solutions for the Gray--Wyner and mutual information regions for DSBS and Gaussian sources, resolving open problems in nonconvex optimization in information theory.

Findings

Analytic expression for DSBS Gray--Wyner region confirmed a 1974 conjecture.

Explicit mutual information region for Gaussian sources derived.

Recovered known expressions for lossy Gray--Wyner regions for DSBS and Gaussian sources.

Abstract

Nonconvex optimization plays a key role in multi-user information theory and related fields, but it is usually difficult to solve. The rate region of the Gray--Wyner source coding system (or almost equivalently, the mutual information region) is a typical example in nonconvex optimization, whose single-letter expression was given by Gray and Wyner. However, due to the nonconvexity of the optimization involved in this expression, previously, there was none nontrivial discrete source for which the analytic expression is known. In this paper, we propose a new strategy to solve nonconvex optimization problems. By this strategy, we provide the analytic expression for the doubly symmetric binary source (DSBS), which confirms positively a conjecture of Gray and Wyner in 1974. We also provide the analytic expression of the mutual information region for the Gaussian source, and provide (or recover) the analytic expressions of the lossy Gray--Wyner region for both the DSBS and Gaussian source. Our proof strategy relies on an auxiliary measure technique and the analytical expression of the optimal-transport divergence region.