A Constructive Method to Maximize Entropy under Marginal Constraints

TL;DR

This paper introduces a constructive method for maximizing Rényi entropy of order 2 under marginal constraints, providing explicit optimal couplings and an efficient iterative algorithm applicable to arbitrary marginals.

Contribution

It offers a new explicit construction of the optimal joint distribution for entropy maximization with marginals, extending beyond previous restrictive feasibility conditions.

Findings

Provides a closed-form optimizer under restrictive conditions

Develops an explicit construction for arbitrary marginals

Designs an iterative algorithm with finite termination

Abstract

We study the problem of maximizing R{\'e}nyi entropy of order $2$ (equivalently, minimizing the index of coincidence) over the set of joint distributions with prescribed marginals. A closed-form optimizer is known under a feasibility condition on the marginals; we show that this condition is highly restrictive. We then provide an explicit construction of an optimal coupling for arbitrary marginals. Our approach characterizes the optimizer's structure and yields an iterative algorithm that terminates in finite time, returning an exact solution after at most $p-1$ updates, where $p$ is the number of rows.