Convergence of Differential Entropies -- II

TL;DR

This paper establishes conditions under which differential entropy converges when probability density functions converge in measure, extending previous results with weaker integrability conditions and providing a complete characterization on bounded domains.

Contribution

It introduces a new entropy-weighted Orlicz condition for entropy convergence, disproves a conjecture by Godavarti and Hero, and characterizes entropy convergence on bounded domains.

Findings

Differential entropy converges under measure convergence with uniform integrability and tightness.

A weaker Orlicz condition suffices for entropy convergence.

On bounded domains, uniform integrability of entropy integrands is necessary and sufficient.

Abstract

We show that under convergence in measure of probability density functions, differential entropy converges whenever the entropy integrands $f_n |\log f_n|$ are uniformly integrable and tight -- a direct consequence of Vitali's convergence theorem. We give an entropy-weighted Orlicz condition: $\sup_n \int f_n\, \Psi(|\log f_n|) < \infty$ for a single superlinear~$\Psi$, strictly weaker than the fixed-$\alpha$ condition of Godavarti and Hero (2004). We also disprove the Godavarti-Hero conjecture that $\alpha > 1$ could be replaced by $\alpha_n \downarrow 1$. We recover the sufficient conditions of Godavarti--Hero, Piera--Parada, and Ghourchian-Gohari-Amini as corollaries. On bounded domains, we prove that uniform integrability of the entropy integrands is both necessary and sufficient -- a complete characterization of entropy convergence.