Law of Log Determinant of Sample Covariance Matrix and Optimal Estimation of Differential Entropy for High-Dimensional Gaussian Distributions

TL;DR

This paper investigates the asymptotic behavior and optimal estimation of the differential entropy and log-determinant of covariance matrices for high-dimensional Gaussian distributions, providing theoretical results and estimators with optimal convergence rates.

Contribution

It establishes a central limit theorem for the log determinant of sample covariance matrices and proposes estimators with optimal convergence in high-dimensional settings.

Findings

Central limit theorem for high-dimensional log determinant

Optimal convergence rates for entropy and log-determinant estimators

Asymptotic minimaxity when dimension grows slower than sample size

Abstract

Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high dimensional setting optimal estimation of the differential entropy and the log-determinant of the covariance matrix. We first establish a central limit theorem for the log determinant of the sample covariance matrix in the high dimensional setting where the dimension $p(n)$ can grow with the sample size $n$. An estimator of the differential entropy and the log determinant is then considered. Optimal rate of convergence is obtained. It is shown that in the case $p(n)/n \rightarrow 0$ the estimator is asymptotically sharp minimax. The ultra-high dimensional setting where $p(n) > n$ is also discussed.