# Bayesian and frequentist approaches to resonance searches

## TL;DR

This paper compares Bayesian and frequentist methods for resonance searches, demonstrating how Bayesian significance tends to be smaller and more conservative than frequentist significance, with implications for interpreting experimental results.

## Contribution

It provides a detailed comparison of Bayesian and frequentist approaches in resonance searches, highlighting the impact of priors and the look-elsewhere effect on significance estimates.

## Key findings

- Bayesian posterior converges to certainty with increasing luminosity.
- Frequentist p-values fluctuate randomly under the background-only hypothesis.
- Bayesian significance is typically 1-2σ smaller than frequentist global significance.

## Abstract

We investigate Bayesian and frequentist approaches to resonance searches using a toy model based on an ATLAS search for the Higgs boson in the diphoton channel. We draw pseudo-data from the background only model and background plus signal model at multiple luminosities, from $10^{-3}$/fb to $10^7$/fb. We chart the change in the Bayesian posterior of the background only model and the global p-value. We find that, as anticipated, the posterior converges to certainty about the model as luminosity increases. The p-value, on the other hand, randomly walks between 0 and 1 if the background only model is true, and otherwise converges to 0. After briefly commenting on the frequentist properties of the posterior, we make a direct comparison of the significances obtained in Bayesian and frequentist frameworks. We find that the well-known look-elsewhere effect reduces local significances by about 1$\sigma$. We furthermore find that significances from our Bayesian framework are typically about 1 to 2$\sigma$ smaller than the global significances, though the reduction depends on the prior, global significance and integrated luminosity. This suggests that even global significances could significantly overstate the evidence against the background only model. We checked that this effect --- the Bayes effect --- was robust with respect to fourteen choices of prior and investigated the Jeffreys-Lindley paradox for three of them.