Multiple testing for signal-agnostic searches of new physics with machine learning

TL;DR

This paper proposes multiple testing strategies to improve signal-agnostic searches for new physics using machine learning, aiming for more uniform anomaly detection across diverse signals.

Contribution

It introduces methods to combine different hypothesis tests, enhancing robustness and uniformity in machine learning-based new physics searches.

Findings

Combining tests achieves performance comparable to the best individual test.

Multiple testing improves uniformity in anomaly detection.

Approaches include p-value combination and test statistic aggregation.

Abstract

In this work, we address the question of how to enhance signal-agnostic searches by leveraging multiple testing strategies. Specifically, we consider hypothesis tests relying on machine learning, where model selection can introduce a bias towards specific families of new physics signals. We show that it is beneficial to combine different tests, characterised by distinct choices of hyperparameters, and that performances comparable to the best available test are generally achieved while providing a more uniform response to various types of anomalies. Focusing on the New Physics Learning Machine, a methodology to perform a signal-agnostic likelihood-ratio test, we explore a number of approaches to multiple testing, such as combining p-values and aggregating test statistics.