Criteria for projected discovery and exclusion sensitivities of counting experiments

TL;DR

This paper critiques the common use of median expected p-values for sensitivity projections in experiments, highlighting flaws and proposing the exact Asimov significance as a better alternative.

Contribution

It introduces the exact Asimov significance $Z^{A}$ as a robust criterion for projected sensitivities, addressing issues with median significance methods.

Findings

Median expected p-values can give counterintuitive sensitivity results.

The exact Asimov significance provides a more reliable sensitivity measure.

Recommendations for standardizing the use of $Z^{A}$ in experimental projections.

Abstract

The projected discovery and exclusion capabilities of particle physics and astrophysics/cosmology experiments are often quantified using the median expected $p$-value or its corresponding significance. We argue that this criterion leads to flawed results, which for example can counterintuitively project lessened sensitivities if the experiment takes more data or reduces its background. We discuss the merits of several alternatives to the median expected significance, both when the background is known and when it is subject to some uncertainty. We advocate for standard use of the "exact Asimov significance" $Z^{\rm A}$ detailed in this paper.