Entropy and the Discrete Central Limit Theorem

TL;DR

This paper proves a strengthened discrete central limit theorem using information theory, showing that the relative entropy between the sum of lattice variables and a discretized Gaussian diminishes as the number of variables grows.

Contribution

It introduces an information-theoretic approach to the discrete CLT, providing elementary proofs and stronger convergence results.

Findings

Relative entropy between sum and discretized Gaussian tends to zero

Method relies solely on information-theoretic tools

Applicable to i.i.d. lattice random variables

Abstract

A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of $n$ independent and identically distributed lattice random variables and an appropriately discretised Gaussian, vanishes as $n\to\infty$.