A new entropy power inequality for integer-valued random variables

TL;DR

This paper introduces a new universal entropy power inequality for sums of independent integer-valued random variables, extending previous results and providing bounds applicable to a broad class of discrete distributions.

Contribution

It presents a novel inequality relating the entropy of sums of integer-valued variables to their individual entropies, applicable universally beyond specific distribution families.

Findings

Established a lower bound on the entropy difference using a universal function g

Extended the inequality to non-identically distributed variables

Included results for conditional entropies

Abstract

The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function provides a sharp inequality $H(X+X')-H(X)\geq 1/2 -o(1)$ when $X,X'$ are i.i.d. with high entropy. This paper provides the inequality $H(X+X')-H(X) \geq g(H(X))$, where $X,X'$ are arbitrary i.i.d. integer-valued random variables and where $g$ is a universal strictly positive function on $\mR_+$ satisfying $g(0)=0$. Extensions to non identically distributed random variables and to conditional entropies are also obtained.