Entropy and thinning of discrete random variables

TL;DR

This paper explores various information-theoretic properties and inequalities related to discrete random variables, focusing on Poisson approximation, entropy behavior, and concentration inequalities, inspired by continuous case analogs.

Contribution

It introduces five types of results connecting information theory and concentration for discrete variables, extending continuous case concepts to the discrete setting.

Findings

Poisson approximation via information-theoretic methods

Maximum entropy property of the Poisson distribution

Discrete concentration inequalities like Poincaré and logarithmic Sobolev

Abstract

We describe five types of results concerning information and concentration of discrete random variables, and relationships between them, motivated by their counterparts in the continuous case. The results we consider are information theoretic approaches to Poisson approximation, the maximum entropy property of the Poisson distribution, discrete concentration (Poincar\'{e} and logarithmic Sobolev) inequalities, monotonicity of entropy and concavity of entropy in the Shepp--Olkin regime.