A Tighter Approximation Guarantee for Greedy Minimum Entropy Coupling

TL;DR

This paper proves that the greedy algorithm for minimum entropy coupling is within approximately 1.44 bits of optimal, improving previous guarantees and establishing tight bounds for the problem.

Contribution

It provides a tighter approximation guarantee for the greedy minimum entropy coupling algorithm and proves the bound is tight.

Findings

Greedy algorithm is within log2(e) bits of optimal.

The entropy of greedy coupling is upper-bounded by H(∧S) + log2(e).

The analysis is tight, with no better constant approximation possible.

Abstract

We examine the minimum entropy coupling problem, where one must find the minimum entropy variable that has a given set of distributions $S = \{p_1, \dots, p_m \}$ as its marginals. Although this problem is NP-Hard, previous works have proposed algorithms with varying approximation guarantees. In this paper, we show that the greedy coupling algorithm of [Kocaoglu et al., AAAI'17] is always within $\log_2(e)$ ($\approx 1.44$) bits of the minimum entropy coupling. In doing so, we show that the entropy of the greedy coupling is upper-bounded by $H\left(\bigwedge S \right) + \log_2(e)$. This improves the previously best known approximation guarantee of $2$ bits within the optimal [Li, IEEE Trans. Inf. Theory '21]. Moreover, we show our analysis is tight by proving there is no algorithm whose entropy is upper-bounded by $H\left(\bigwedge S \right) + c$ for any constant $c<\log_2(e)$. Additionally, we examine a special class of instances where the greedy coupling algorithm is exactly optimal.