AI-Enabled Operations at Fermi Complex: Multivariate Time Series Prediction for Outage Prediction and Diagnosis

TL;DR

This paper presents an AI framework using deep learning and random forest models to predict and diagnose beam outages in Fermilab's accelerator complex, aiming to reduce operational downtime and improve decision-making.

Contribution

It evaluates multiple AI architectures for outage prediction and introduces a random forest-based system for automated, confidence-scored outage labeling, addressing limitations of existing threshold-based alarms.

Findings

Deep learning models show varying effectiveness in outage prediction.

Random Forest provides consistent, confidence-scored outage labels.

Identifies key gaps for AI to enable predictive outage management.

Abstract

The Main Control Room of the Fermilab accelerator complex continuously gathers extensive time-series data from thousands of sensors monitoring the beam. However, unplanned events such as trips or voltage fluctuations often result in beam outages, causing operational downtime. This downtime not only consumes operator effort in diagnosing and addressing the issue but also leads to unnecessary energy consumption by idle machines awaiting beam restoration. The current threshold-based alarm system is reactive and faces challenges including frequent false alarms and inconsistent outage-cause labeling. To address these limitations, we propose an AI-enabled framework that leverages predictive analytics and automated labeling. Using data from $2,703$ Linac devices and $80$ operator-labeled outages, we evaluate state-of-the-art deep learning architectures, including recurrent, attention-based, and linear models, for beam outage prediction. Additionally, we assess a Random Forest-based labeling system for providing consistent, confidence-scored outage annotations. Our findings highlight the strengths and weaknesses of these architectures for beam outage prediction and identify critical gaps that must be addressed to fully harness AI for transitioning downtime handling from reactive to predictive, ultimately reducing downtime and improving decision-making in accelerator management.