On the entropy and information of Gaussian mixtures

TL;DR

This paper proves convexity properties of entropy and Fisher information for Gaussian mixtures, confirming conjectures and extending previous results, with applications to convergence rates in the central limit theorem.

Contribution

It establishes new convexity results for entropy and Fisher information of Gaussian mixtures, confirming conjectures and extending prior work.

Findings

Entropy of Gaussian mixtures is concave in mixing parameter t.

Fisher information matrix is operator convex for Gaussian mixtures.

Rates of convergence for Fisher information in the CLT are derived.

Abstract

We establish several convexity properties for the entropy and Fisher information of mixtures of centered Gaussian distributions. First, we prove that if $X_1, X_2$ are independent scalar Gaussian mixtures, then the entropy of $\sqrt{t}X_1 + \sqrt{1-t}X_2$ is concave in $t \in [0,1]$, thus confirming a conjecture of Ball, Nayar and Tkocz (2016) for this class of random variables. In fact, we prove a generalisation of this assertion which also strengthens a result of Eskenazis, Nayar and Tkocz (2018). For the Fisher information, we extend a convexity result of Bobkov (2022) by showing that the Fisher information matrix is operator convex as a matrix-valued function acting on densities of mixtures in $\mathbb{R}^d$. As an application, we establish rates for the convergence of the Fisher information matrix of the sum of weighted i.i.d. Gaussian mixtures in the operator norm along the central limit theorem under mild moment assumptions.