Evidence for Conservatism in LHC SUSY Searches

TL;DR

This paper analyzes LHC SUSY search results and finds evidence that experimental collaborations may be overly conservative in their uncertainty estimates, as indicated by an excess of medium significance p-values.

Contribution

It provides the first statistical analysis of p-value distributions in LHC SUSY searches, revealing potential over-conservatism in uncertainty estimation.

Findings

Significant excess of medium sigma p-values at p=0.005

Indication of over-conservatism in SUSY search analyses

Highlights the need for reassessment of uncertainty estimation methods

Abstract

The standard in the high energy physics community for claiming discovery of new physics is a $5\sigma$ excess in the observed signal over the estimated background. While a $3\sigma$ excess is not enough to claim discovery, it is certainly enough to pique the interest of both experimentalists and theorists. However, with a large number of searches performed by both the ATLAS and CMS collaborations at the LHC, one expects a nonzero number of multi-$\sigma$ results simply due to statistical fluctuations in the no-signal scenario. Our analysis examines the distribution of p-values for CMS and ATLAS supersymmetry (SUSY) searches using the full 2011 data set to determine if the collaborations are being overly conservative in their analyses. We find that there is a statistically significant excess of `medium' $\sigma$ values at the level of $p=0.005$, indicating over-conservativism in the estimation of uncertainties.