Reinforcement Learning for Accelerator Beamline Control: a simulation-based approach

TL;DR

This paper presents RLABC, a reinforcement learning library that automates beamline optimization in particle accelerators using simulation, achieving high transmission efficiency comparable to expert manual tuning.

Contribution

RLABC introduces a simulation-based RL framework for beamline optimization, integrating accelerator physics with machine learning to automate and improve tuning processes.

Findings

Achieved 94% and 91% transmission rates on two beamlines.

Demonstrated effectiveness of DDPG in accelerator control tasks.

Provided a versatile tool bridging physics and RL research.

Abstract

Particle accelerators play a pivotal role in advancing scientific research, yet optimizing beamline configurations to maximize particle transmission remains a labor-intensive task requiring expert intervention. In this work, we introduce RLABC (Reinforcement Learning for Accelerator Beamline Control), a Python-based library that reframes beamline optimization as a reinforcement learning (RL) problem. Leveraging the Elegant simulation framework, RLABC automates the creation of an RL environment from standard lattice and element input files, enabling sequential tuning of magnets to minimize particle losses. We define a comprehensive state representation capturing beam statistics, actions for adjusting magnet parameters, and a reward function focused on transmission efficiency. Employing the Deep Deterministic Policy Gradient (DDPG) algorithm, we demonstrate RLABC's efficacy on two beamlines, achieving transmission rates of 94% and 91%, comparable to expert manual optimizations. This approach bridges accelerator physics and machine learning, offering a versatile tool for physicists and RL researchers alike to streamline beamline tuning.