Thinning, Entropy and the Law of Thin Numbers

TL;DR

This paper explores the properties of Renyi's thinning operation on discrete variables, establishing new convergence rates and bounds related to Poisson approximation, and introduces a thinning Markov chain analogous to Gaussian processes.

Contribution

It introduces a thinning limit theorem for convolutions of discrete distributions, providing convergence rates and nonasymptotic bounds, and develops a thinning Markov chain with properties similar to Gaussian processes.

Findings

Thinning operation relates to Poisson approximation in discrete distributions.

A rate of convergence for the thinning limit theorem is established.

A thinning Markov chain analogous to the Ornstein-Uhlenbeck process is introduced.

Abstract

Renyi's "thinning" operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in connection with information-theoretic inequalities related to Poisson approximation results. The classical Binomial-to-Poisson convergence (sometimes referred to as the "law of small numbers" is seen to be a special case of a thinning limit theorem for convolutions of discrete distributions. A rate of convergence is provided for this limit, and nonasymptotic bounds are also established. This development parallels, in part, the development of Gaussian inequalities leading to the information-theoretic version of the central limit theorem. In particular, a "thinning Markov chain" is introduced, and it is shown to play a role analogous to that of the Ornstein-Uhlenbeck process in connection to the entropy power inequality.