Quantifying lost information due to covariance matrix estimation in parameter inference

TL;DR

This paper quantifies the information loss in parameter inference caused by estimated covariance matrices, develops methods to assess and mitigate this loss, and applies these to real and simulated cosmological data.

Contribution

It introduces a framework to quantify and restore information loss due to covariance estimation, including new estimators and practical guidelines for simulation requirements.

Findings

Detected 10% information loss in DES weak lensing data.

Found no significant information loss in KiDS data.

Estimated simulation counts needed for Euclid-like surveys to limit information loss.

Abstract

Parameter inference with an estimated covariance matrix systematically loses information due to the remaining uncertainty of the covariance matrix. Here, we quantify this loss of precision and develop a framework to hypothetically restore it, which allows to judge how far away a given analysis is from the ideal case of a known covariance matrix. We point out that it is insufficient to estimate this loss by debiasing a Fisher matrix as previously done, due to a fundamental inequality that describes how biases arise in non-linear functions. We therefore develop direct estimators for parameter credibility contours and the figure of merit. We apply our results to DES Science Verification weak lensing data, detecting a 10% loss of information that increases their credibility contours. No significant loss of information is found for KiDS. For a Euclid-like survey, with about 10 nuisance parameters we find that 2900 simulations are sufficient to limit the systematically lost information to 1%, with an additional uncertainty of about 2%. Without any nuisance parameters 1900 simulations are sufficient to only lose 1% of information. We also derive an estimator for the Fisher matrix of the unknown true covariance matrix, two estimators of its inverse with different physical meanings, and an estimator for the optimally achievable figure of merit. The formalism here quantifies the gains to be made by running more simulated datasets, allowing decisions to be made about numbers of simulations in an informed way.