On Focusing Statistical Power for Searches and Measurements in Particle Physics

TL;DR

This paper introduces a new test statistic that enhances statistical power focus in particle physics searches, demonstrated on Higgs and dark matter datasets, with machine learning aiding confidence interval construction.

Contribution

It proposes an alternative test statistic that concentrates statistical power on physics-motivated regions, improving upon the traditional likelihood ratio test in composite hypothesis testing.

Findings

Improved test statistic focuses power on relevant parameter regions.

Demonstrated better performance on Higgs and dark matter datasets.

Machine learning enables efficient Neyman construction for valid confidence intervals.

Abstract

Particle physics experiments rely on the (generalised) likelihood ratio test (LRT) for searches and measurements, which consist of composite hypothesis tests. However, this test is not guaranteed to be optimal, as the Neyman-Pearson lemma pertains only to simple hypothesis tests. Any choice of test statistic thus implicitly determines how statistical power varies across the parameter space. An improvement in the core statistical testing methodology for general settings with composite tests would have widespread ramifications across experiments. We discuss an alternate test statistic that provides the data analyzer an ability to focus the power of the test on physics-motivated regions of the parameter space. We demonstrate the improvement from this technique compared to the LRT on a Higgs $\rightarrow\tau\tau$ dataset simulated by the ATLAS experiment and a dark matter dataset inspired by the LZ experiment. We also employ machine learning to efficiently perform the Neyman construction, which is essential to ensure statistically valid confidence intervals.