Non-linear shrinkage estimation of large-scale structure covariance

TL;DR

This paper demonstrates that a non-linear shrinkage estimator can significantly improve covariance matrix estimation in large-scale structure cosmology, reducing the number of required mock realisations by a factor of about 50 without prior data assumptions.

Contribution

It applies and optimizes a non-linear shrinkage estimator for covariance matrices in cosmology, achieving high accuracy with fewer mock realisations and negligible computational cost.

Findings

Reduces the number of mock realisations needed by a factor of ~50.

Achieves comparable bias and variance to standard estimators.

Operates without prior information on data or covariance structure.

Abstract

In many astrophysical settings covariance matrices of large datasets have to be determined empirically from a finite number of mock realisations. The resulting noise degrades inference and precludes it completely if there are fewer realisations than data points. This work applies a recently proposed non-linear shrinkage estimator of covariance to a realistic example from large-scale structure cosmology. After optimising its performance for the usage in likelihood expressions, the shrinkage estimator yields subdominant bias and variance comparable to that of the standard estimator with a factor $\sim 50$ less realisations. This is achieved without any prior information on the properties of the data or the structure of the covariance matrix, at negligible computational cost.