Entropic Conditional Central Limit Theorem and Hadamard Compression

TL;DR

This paper introduces an entropic approach to establish a stronger conditional central limit theorem and analyzes Hadamard compression, demonstrating that most output distributions tend toward Gaussianity and are conditionally insensitive.

Contribution

The paper develops an entropic conditional CLT and applies it to Hadamard transforms, providing a theoretical foundation for Hadamard compression and Gaussianity measurement.

Findings

Conditional entropy measures asymptotic Gaussianity.

Most Hadamard-transformed outputs tend to Gaussian distribution.

Conditional Fisher information can quantify Gaussianity.

Abstract

We make use of an entropic property to establish a convergence theorem (Main Theorem), which reveals that the conditional entropy measures the asymptotic Gaussianity. As an application, we establish the {\it entropic conditional central limit theorem} (CCLT), which is stronger than the classical CCLT. As another application, we show that continuous input under iterated Hadamard transform, almost every distribution of the output conditional on the values of the previous signals will tend to Gaussian, and the conditional distribution is in fact insensitive to the condition. The results enable us to make a theoretic study concerning Hadamard compression, which provides a solid theoretical analysis supporting the simulation results in previous literature. We show also that the conditional Fisher information can be used to measure the asymptotic Gaussianity.