Comment on "An implementation of neural simulation-based inference for parameter estimation in ATLAS''

TL;DR

This paper critiques a neural simulation-based inference method used in ATLAS, demonstrating through a toy example that the proposed double-bootstrapping technique fails to accurately estimate uncertainties.

Contribution

It provides a critical analysis showing that the existing double-bootstrapping approach does not effectively quantify uncertainties in neural simulation-based inference.

Findings

Double-bootstrapping does not capture uncertainties accurately.

Toy example demonstrates the method's limitations.

Highlights need for improved uncertainty estimation techniques.

Abstract

The paper titled "An implementation of neural simulation-based inference for parameter estimation in ATLAS" by the ATLAS collaboration (arXiv:2412.01600v1 [hep-ex]) describes the implementation of neural simulation-based inference for a measurement analysis performed by ATLAS. The uncertainties in the analysis arising from the finiteness of the simulated datasets are estimated using a novel double-bootstrapping technique described in that work. In the present comment, it is claimed and demonstrated, using a toy example, that the double-bootstrapping technique does not actually capture the aforementioned uncertainties.