Biased rate estimates in bump-hunt searches

TL;DR

This paper discusses how the common bump-hunt method for discovering new particles can lead to biased rate estimates and underestimated uncertainties, especially when the particle's mass and width are unknown, impacting significance calculations.

Contribution

It quantifies the bias and uncertainties introduced by the 'greedy' bump-hunt approach and provides corrections for more accurate significance and rate estimates in particle searches.

Findings

3σ evidence likely overestimates true rate by ~10%

Mass uncertainty underestimated by ~20%

Additional data needed for 5σ discovery can be hundreds of inverse femtobarns

Abstract

The cleanest way to discover a new particle is generally the "bump-hunt" methodology: looking for a localised excess in a mass (or related) distribution. However, if the mass of the particle being discovered is not known the procedure of quoting the most significant excess seen is "greedy", and random fluctuations from the background can be merged with the signal. This means that an observed 3{\sigma} evidence probably has the rate over-estimated by of order 10% and the mass uncertainty underestimated by more like 20%. If the width of the particle being discovered is also unknown, or the experimental resolution uncertain, the effect grows larger. In the context of LHC, the data-doubling period is now measured in years. If evidence for a genuine signal is obtained at the 3{\sigma} level, the true signal rate probably corresponds to a lower significance. The additional data required to produce a 5{\sigma} discovery may be hundreds of inverse femtobarn more than might be naively expected.