On hypothesis testing, trials factor, hypertests and the BumpHunter

TL;DR

This paper discusses hypothesis testing, introduces hypertests to address the trials factor, and presents BumpHunter as a practical tool for detecting signals in high energy physics data, exemplified through real-world applications.

Contribution

It introduces the concept of hypothesis hypertests to properly account for the trials factor and demonstrates the effectiveness of BumpHunter in high energy physics searches.

Findings

BumpHunter effectively detects localized excesses in data.

Hypertests improve the reliability of significance estimates.

Application to real data shows practical utility in physics searches.

Abstract

A detailed presentation of hypothesis testing is given. The "look elsewhere" effect is illustrated, and a treatment of the trials factor is proposed with the introduction of hypothesis hypertests. An example of such a hypertest is presented, named BumpHunter, which is used in the recent ATLAS dijet resonance search, and in an earlier version in the CDF Global Search, to look for exotic phenomena in high energy physics. As a demonstration, the BumpHunter is used to address Problem 1 of the Banff Challenge.