On the entropy growth of sums of iid discrete random variables

TL;DR

This paper establishes an asymptotic lower bound on the Shannon entropy of sums of iid discrete variables, based on a new measure called incommensurability rank, without relying on central limit theorems.

Contribution

It introduces the incommensurability rank and derives a novel entropy growth lower bound for sums of iid discrete variables.

Findings

Entropy grows at least as fast as (r(X)/2) * log(N) plus a constant

The bound applies to arbitrary iid discrete variables, not just lattice cases

The derivation avoids reliance on central limit theorems

Abstract

We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of the random variable -- a positive integer quantity that we introduce. The derivation does not rely on central limit theorems, but builds upon the known expressions of the asymptotic entropy of the multinomial distribution and sums of iid lattice random variables, which correspond to the case $r(X)=1$.