Sharp Bounds on the Entropy of the Poisson Law and Related Quantities

TL;DR

This paper provides simple, tight bounds on the entropy of the Poisson distribution and related quantities, facilitating capacity calculations for Poisson channels and improving existing bounds with accessible methods.

Contribution

It introduces asymptotically tight, easy-to-compute bounds on Poisson entropy and related divergences, enhancing prior asymptotic expansions and bounds in information theory.

Findings

Derived asymptotically tight bounds on H(λ)

Refined bounds on the divergence between binomial and Poisson distributions

Provided bounds on the entropy of the binomial distribution

Abstract

One of the difficulties in calculating the capacity of certain Poisson channels is that H(lambda), the entropy of the Poisson distribution with mean lambda, is not available in a simple form. In this work we derive upper and lower bounds for H(lambda) that are asymptotically tight and easy to compute. The derivation of such bounds involves only simple probabilistic and analytic tools. This complements the asymptotic expansions of Knessl (1998), Jacquet and Szpankowski (1999), and Flajolet (1999). The same method yields tight bounds on the relative entropy D(n, p) between a binomial and a Poisson, thus refining the work of Harremoes and Ruzankin (2004). Bounds on the entropy of the binomial also follow easily.