Efficient Approximate Minimum Entropy Coupling of Multiple Probability Distributions

TL;DR

This paper introduces an efficient algorithm that approximates the minimum entropy coupling of multiple probability distributions within 2 bits, applicable even for infinite supports, with potential uses in various information theory applications.

Contribution

The paper presents a novel polynomial-time algorithm for approximate minimum entropy coupling within 2 bits of optimal, extending to infinite distributions and supports.

Findings

Algorithm achieves within 2 bits of optimal entropy

Applicable to infinite distributions and supports

Potential applications in random number generation and causal inference

Abstract

Given a collection of probability distributions $p_{1},\ldots,p_{m}$, the minimum entropy coupling is the coupling $X_{1},\ldots,X_{m}$ ($X_{i}\sim p_{i}$) with the smallest entropy $H(X_{1},\ldots,X_{m})$. While this problem is known to be NP-hard, we present an efficient algorithm for computing a coupling with entropy within 2 bits from the optimal value. More precisely, we construct a coupling with entropy within 2 bits from the entropy of the greatest lower bound of $p_{1},\ldots,p_{m}$ with respect to majorization. This construction is also valid when the collection of distributions is infinite, and when the supports of the distributions are infinite. Potential applications of our results include random number generation, entropic causal inference, and functional representation of random variables.