Compound Poisson Approximation via Information Functionals

TL;DR

This paper develops an information-theoretic framework for compound Poisson approximation, providing nonasymptotic bounds on distribution distances using novel information functionals similar to Fisher information.

Contribution

It introduces two new information functionals and derives bounds for compound Poisson approximation, extending previous Gaussian and Poisson approximation techniques.

Findings

Derived bounds on total variation and relative entropy distances.

Introduced two information functionals analogous to Fisher information.

Provided detailed comparisons with existing bounds.

Abstract

An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of independent integer-valued random variables $Y_i$. Nonasymptotic bounds are derived for the distance between $P_{S_n}$ and an appropriately chosen compound Poisson law. In the case where all $Y_i$ have the same conditional distribution given $\{Y_i\neq 0\}$, a bound on the relative entropy distance between $P_{S_n}$ and the compound Poisson distribution is derived, based on the data-processing property of relative entropy and earlier Poisson approximation results. When the $Y_i$ have arbitrary distributions, corresponding bounds are derived in terms of the total variation distance. The main technical ingredient is the introduction of two "information functionals," and the analysis of their properties. These information functionals play a role analogous to that of the classical Fisher information in normal approximation. Detailed comparisons are made between the resulting inequalities and related bounds.