Monotonicity, thinning and discrete versions of the Entropy Power Inequality

TL;DR

This paper explores the properties of entropy for sums of discrete random variables, proposing a new inequality that holds universally, and investigates the effects of thinning on entropy, drawing parallels with continuous scaling.

Contribution

It introduces a novel discrete Entropy Power Inequality that always holds and establishes a sharp bound on entropy behavior under thinning for ultra log-concave variables.

Findings

A new discrete Entropy Power Inequality that is universally valid.

A sharp entropy bound for ultra log-concave variables under thinning.

A stronger form of entropy concavity analogous to continuous scaling.

Abstract

We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the Entropy Power Inequality do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon's Entropy Power Inequality is the behaviour of entropy on scaling of continuous random variables. We believe that R\'{e}nyi's operation of thinning discrete random variables plays a similar role to scaling, and give a sharp bound on how the entropy of ultra log-concave random variables behaves on thinning. In the spirit of the monotonicity results established by Artstein, Ball, Barthe and Naor, we prove a stronger version of concavity of entropy, which implies a strengthened form of our discrete Entropy Power Inequality.