Optimal Covariance Cleaning for Heavy-Tailed Distributions: Insights from Information Theory

TL;DR

This paper explores optimal covariance estimation for heavy-tailed distributions, revealing that minimizing Frobenius norm aligns with minimizing information loss asymptotically, with implications for practical applications in physics and finance.

Contribution

It establishes a connection between covariance estimation and information theory for heavy-tailed distributions, extending random matrix theory applicability beyond Gaussian cases.

Findings

Frobenius norm minimization aligns with information loss minimization asymptotically.

Deviations in finite matrices for Student's t distributions vanish as size increases.

Results extend random matrix theory to heavy-tailed distributions like Student's t.

Abstract

In optimal covariance cleaning theory, minimizing the Frobenius norm between the true population covariance matrix and a rotational invariant estimator is a key step. This estimator can be obtained asymptotically for large covariance matrices, without knowledge of the true covariance matrix. In this study, we demonstrate that this minimization problem is equivalent to minimizing the loss of information between the true population covariance and the rotational invariant estimator for normal multivariate variables. However, for Student's t distributions, the minimal Frobenius norm does not necessarily minimize the information loss in finite-sized matrices. Nevertheless, such deviations vanish in the asymptotic regime of large matrices, which might extend the applicability of random matrix theory results to Student's t distributions. These distributions are characterized by heavy tails and are frequently encountered in real-world applications such as finance, turbulence, or nuclear physics. Therefore, our work establishes a connection between statistical random matrix theory and estimation theory in physics, which is predominantly based on information theory.