Anomaly Detection for Automated Data Quality Monitoring in the CMS Detector

TL;DR

AutoDQM employs advanced statistical and machine learning techniques to detect anomalies in CMS detector data, significantly improving data quality assessment for large particle physics experiments.

Contribution

The paper introduces AutoDQM, a novel automated system combining statistical and machine learning methods for real-time data quality monitoring in particle detectors.

Findings

AutoDQM detects anomalies 4-6 times more effectively than previous methods.

It successfully identifies data affected by detector malfunctions.

The system is applicable to large-scale particle physics data sets.

Abstract

Successful operation of large particle detectors like the Compact Muon Solenoid (CMS) at the CERN Large Hadron Collider requires rapid, in-depth assessment of data quality. We introduce the ``AutoDQM'' system for Automated Data Quality Monitoring using advanced statistical techniques and unsupervised machine learning. Anomaly detection algorithms based on the beta-binomial probability function, principal component analysis, and neural network autoencoder image evaluation are tested on the full set of proton-proton collision data collected by CMS in 2022. AutoDQM identifies anomalous ``bad'' data affected by significant detector malfunction at a rate 4 -- 6 times higher than ``good'' data, demonstrating its effectiveness as a general data quality monitoring tool.