On the Distribution of the Information Density of Gaussian Random Vectors: Explicit Formulas and Tight Approximations

TL;DR

This paper derives explicit formulas and tight approximations for the distribution of information density in Gaussian vectors, enabling efficient numerical calculations and analyzing the validity of Gaussian approximations.

Contribution

It provides new series representations, closed-form expressions, and recurrence formulas for the distribution of information density in Gaussian vectors.

Findings

Derived series representations for PDF and CDF of information density.

Provided closed-form formulas for key special cases.

Developed efficient numerical methods with high accuracy.

Abstract

Based on the canonical correlation analysis we derive series representations of the probability density function (PDF) and the cumulative distribution function (CDF) of the information density of arbitrary Gaussian random vectors as well as a general formula to calculate the central moments. Using the general results we give closed-form expressions of the PDF and CDF and explicit formulas of the central moments for important special cases. Furthermore, we derive recurrence formulas and tight approximations of the general series representations, which allow very efficient numerical calculations with an arbitrarily high accuracy as demonstrated with an implementation in Python publicly available on GitLab. Finally, we discuss the (in)validity of Gaussian approximations of the information density.