An efficiency upper bound for inverse covariance estimation

TL;DR

This paper establishes a fundamental lower bound on the number of samples needed to accurately estimate entries of the inverse covariance matrix in high-dimensional Gaussian distributions, highlighting intrinsic limitations in statistical efficiency.

Contribution

The authors derive an upper bound for estimation efficiency and demonstrate the minimal sample size required for reliable inference of inverse covariance entries in high dimensions.

Findings

Estimating off-diagonal entries requires at least proportional to dimension samples.

With fewer samples, certain correlation hypotheses become indistinguishable.

The results set fundamental limits on inverse covariance estimation in high-dimensional settings.

Abstract

We derive an upper bound for the efficiency of estimating entries in the inverse covariance matrix of a high dimensional distribution. We show that in order to approximate an off-diagonal entry of the density matrix of a $d$-dimensional Gaussian random vector, one needs at least a number of samples proportional to $d$. Furthermore, we show that with $n \ll d$ samples, the hypothesis that two given coordinates are fully correlated, when all other coordinates are conditioned to be zero, cannot be told apart from the hypothesis that the two are uncorrelated.