The Banff Challenge: Statistical Detection of a Noisy Signal

TL;DR

This paper introduces statistical methods for detecting signals in noisy data from particle physics experiments, comparing frequentist and Bayesian approaches, and advocating for the use of p-value functions for inference.

Contribution

It develops simple probability models and derives both frequentist and Bayesian procedures for signal detection, highlighting the effectiveness of the p-value function for inference.

Findings

Both methods are highly accurate in realistic scenarios.

Frequentist procedures are better for interval estimation.

Bayesian methods provide slightly better point estimates.

Abstract

Particle physics experiments such as those run in the Large Hadron Collider result in huge quantities of data, which are boiled down to a few numbers from which it is hoped that a signal will be detected. We discuss a simple probability model for this and derive frequentist and noninformative Bayesian procedures for inference about the signal. Both are highly accurate in realistic cases, with the frequentist procedure having the edge for interval estimation, and the Bayesian procedure yielding slightly better point estimates. We also argue that the significance, or $p$-value, function based on the modified likelihood root provides a comprehensive presentation of the information in the data and should be used for inference.