Fisher's Mirage: Noise Tightening of Cosmological Constraints in Simulation-Based Inference

TL;DR

This paper investigates how statistical noise in numerical derivatives affects Fisher forecasts in large-scale structure surveys, revealing that noise can bias constraints and proposing methods to recover true limits.

Contribution

It provides a detailed analysis of noise effects on Fisher forecasts and offers techniques to correct for noise bias in simulation-based inference.

Findings

Noisy derivatives can bias Fisher constraints, making them artificially tighter.

Higher-order differentiation schemes are more susceptible to noise effects.

Methods to recover true noise-free constraints from noisy Fisher forecasts.

Abstract

We systematically analyze the implications of statistical noise within numerical derivatives on simulation-based Fisher forecasts for large scale structure surveys. Noisy numerical derivatives resulting from a finite number of simulations, $N_{sims}$, act to bias the associated Fisher forecast such that the resulting marginalized constraints can be significantly tighter than the noise-free limit. We show the source of this effect can be traced to the influence of the noise on the marginalization process. Parameters such as the neutrino mass, $\M$, for which higher-order forward differentiation schemes are commonly used, are more prone to noise; the predicted constraints can be akin to those purely from a random instance of statistical noise even using $(1\mathrm{Gpc}/h)^{3}$ simulations with $N_{sims}=500$ realizations. We demonstrate how derivative noise can artificially reduce parameter degeneracies and seemingly null the effects of adding nuisance parameters to the forecast, such as HOD fitting parameters. We mathematically characterize these effects through a full statistical analysis, and demonstrate how confidence intervals for the true noise-free, $N_{sims} \rightarrow \infty$, Fisher constraints can be recovered even when noise comprises a consequential component of the measured signal. The findings and approaches developed here are important for ensuring simulation-based analyses can be accurately used to assess upcoming survey capabilities.