DINAMO: Dynamic and INterpretable Anomaly MOnitoring for Large-Scale Particle Physics Experiments

TL;DR

DINAMO introduces an interpretable, scalable anomaly detection framework for large-scale particle physics experiments, combining statistical and machine learning methods to improve data quality monitoring and reduce reliance on human operators.

Contribution

The paper presents DINAMO, a novel DQM framework that uses evolving histogram templates with uncertainties and incorporates a transformer-based ML model for enhanced anomaly detection.

Findings

High accuracy demonstrated on synthetic datasets

Statistical variant adopted by LHCb experiment

Framework offers interpretability and adaptability

Abstract

Ensuring reliable data collection in large-scale particle physics experiments demands Data Quality Monitoring (DQM) procedures to detect possible detector malfunctions and preserve data integrity. Traditionally, this resource-intensive task has been handled by human shifters who struggle with frequent changes in operational conditions. We present DINAMO: a novel, interpretable, robust, and scalable DQM framework designed to automate anomaly detection in time-dependent settings. Our approach constructs evolving histogram templates with built-in uncertainties, featuring both a statistical variant - extending the classical Exponentially Weighted Moving Average (EWMA) - and a machine learning (ML)-enhanced version that leverages a transformer encoder for improved adaptability. Experimental validations on synthetic datasets demonstrate the high accuracy, adaptability, and interpretability of these methods. The statistical variant is being commissioned in the LHCb experiment at the Large Hadron Collider, underscoring its real-world impact. The code used in this study is available at https://github.com/ArseniiGav/DINAMO.