Developing Robust Digital Twins and Reinforcement Learning for Accelerator Control Systems at the Fermilab Booster

TL;DR

This paper presents the development of a digital twin and reinforcement learning approach for precise control of the Fermilab Booster's magnet power supply, enhancing regulation accuracy through machine learning models validated for real-world deployment.

Contribution

It introduces a validated digital twin of the Booster-GMPS system and demonstrates reinforcement learning for adaptive control under realistic conditions.

Findings

Successful creation of a digital twin using LSTM

Deployment of a DQN policy for GMPS regulation

Effective regulation under time-varying perturbations

Abstract

We describe the offline machine learning (ML) development for an effort to precisely regulate the Gradient Magnet Power Supply (GMPS) at the Fermilab Booster accelerator complex via a Field-Programmable Gate Array (FPGA). As part of this effort, we created a digital twin of the Booster-GMPS control system by training a Long Short-Term Memory (LSTM) to capture its full dynamics. We outline the path we took to carefully validate our digital twin before deploying it as a reinforcement learning (RL) environment. Additionally, we demonstrate the use of a Deep Q-Network (DQN) policy model with the capability to regulate the GMPS against realistic time-varying perturbations.