Sensitivity to New Physics Phenomena in Anomaly Detection: A Study of Untunable Hyperparameters

TL;DR

This paper evaluates how sensitive various anomaly detection methods are to untunable hyperparameters in collider physics, proposing a permutation test for robust signal detection.

Contribution

It systematically compares the sensitivity of four semi-supervised anomaly detection methods to hyperparameters and introduces a permutation test for improved statistical assessment.

Findings

Sensitivity varies significantly with hyperparameters

Permutation test enhances robustness of anomaly detection

Auto-Encoders and Isolation Forest show promising results

Abstract

The search for physics beyond the Standard Model (BSM) at collider experiments requires model-independent strategies to avoid missing possible discoveries of unexpected signals. Anomaly detection (AD) techniques offer a promising approach by identifying deviations from the Standard Model (SM) and have been extensively studied. The sensitivity of these methods to untunable hyperparameters has not been systematically compared, however. This study addresses it by investigating four semi-supervised AD methods -- Auto-Encoders, Deep Support Vector Data Description, Histogram-based Outlier Score, and Isolation Forest -- trained on simulated SM background events. In this paper, we study the sensitivity of these methods to BSM benchmark signals as a function of these untunable hyperparameters. Such a study is complemented by a proposal of a non-parametric permutation test using signal-agnostic statistics, which can provide a robust statistical assessment.