Minimal Joint Entropy and Order-Preserving Couplings

TL;DR

This paper investigates the extremal values of Shannon entropy for joint distributions with fixed marginals, revealing that minimum entropy couplings are order-preserving and maximum entropy couplings are independent, providing insights into system disorder.

Contribution

It proves that the minimal entropy coupling must be order-preserving, a novel characterization in the study of joint distributions with fixed marginals.

Findings

Minimum-entropy coupling is order-preserving.

Maximum-entropy coupling is the independent distribution.

Entropy measures system disorder in joint distributions.

Abstract

This paper focuses on the extreme-value problem for Shannon entropy of the joint distribution with given marginals. It is proved that the minimum-entropy coupling must be of order-preserving, while the maximum-entropy coupling coincides with the independent one. Note that in this sense, we interpret entropy as a measure of system disorder.