Explainable physics-based constraints on reinforcement learning for accelerator controls

TL;DR

This paper introduces an explainable reinforcement learning framework for particle accelerator control that incorporates physics-based constraints, enhancing transparency, trust, and reliable convergence in complex environments.

Contribution

It proposes a novel RL approach using learnable physics-based surrogate functions, either neural or sparse, to improve interpretability and performance in accelerator control tasks.

Findings

Learned physics constraints match established physics principles.

Physics-based surrogate improves RL convergence in high-dimensional environments.

The framework increases transparency and trust in accelerator control decisions.

Abstract

We present a reinforcement learning (RL) framework for controlling particle accelerator experiments that builds explainable physics-based constraints on agent behavior. The goal is to increase transparency and trust by letting users verify that the agent's decision-making process incorporates suitable physics. Our algorithm uses a learnable surrogate function for physical observables, such as energy, and uses them to fine-tune how actions are chosen. This surrogate can be represented by a neural network or by an interpretable sparse dictionary model. We test our algorithm on a range of particle accelerator controls environments designed to emulate the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment. In addition, we find that the introduction of a physics-based surrogate enables our reinforcement learning algorithms to reliably converge for difficult high-dimensional accelerator controls environments.