Autoregressive Transformers for Disruption Prediction in Nuclear Fusion Plasmas

TL;DR

This paper explores autoregressive transformer models to predict plasma disruptions in nuclear fusion tokamaks, achieving improved accuracy and revealing the plasma's persistent memory effect.

Contribution

It introduces variations of masked autoregressive transformers for disruption prediction and compares them to neural networks to understand plasma memory.

Findings

Transformers outperform existing methods by 5% in ROC AUC.

Models reveal the plasma's persistent memory influenced by tokamak controls.

Autoregressive transformers enhance disruption prediction accuracy.

Abstract

The physical sciences require models tailored to specific nuances of different dynamics. In this work, we study outcome predictions in nuclear fusion tokamaks, where a major challenge are \textit{disruptions}, or the loss of plasma stability with damaging implications for the tokamak. Although disruptions are difficult to model using physical simulations, machine learning (ML) models have shown promise in predicting these phenomena. Here, we first study several variations on masked autoregressive transformers, achieving an average of 5\% increase in Area Under the Receiving Operating Characteristic metric above existing methods. We then compare transformer models to limited context neural networks in order to shed light on the ``memory'' of plasma effected by tokamaks controls. With these model comparisons, we argue for the persistence of a memory throughout the plasma \textit{in the context of tokamaks} that our model exploits.