Plasma Shape Control via Zero-shot Generative Reinforcement Learning

TL;DR

This paper introduces a zero-shot reinforcement learning framework that leverages offline data and generative imitation learning to achieve adaptable plasma shape control without task-specific retraining.

Contribution

It combines GAIL with Hilbert space representation learning to create a versatile, goal-oriented control policy from offline data, enabling zero-shot deployment for plasma shape control.

Findings

Policy accurately tracks plasma shape trajectories in simulations.

Method outperforms traditional controllers in stability and precision.

Framework demonstrates potential for future fusion reactor control systems.

Abstract

Traditional PID controllers have limited adaptability for plasma shape control, and task-specific reinforcement learning (RL) methods suffer from limited generalization and the need for repetitive retraining. To overcome these challenges, this paper proposes a novel framework for developing a versatile, zero-shot control policy from a large-scale offline dataset of historical PID-controlled discharges. Our approach synergistically combines Generative Adversarial Imitation Learning (GAIL) with Hilbert space representation learning to achieve dual objectives: mimicking the stable operational style of the PID data and constructing a geometrically structured latent space for efficient, goal-directed control. The resulting foundation policy can be deployed for diverse trajectory tracking tasks in a zero-shot manner without any task-specific fine-tuning. Evaluations on the HL-3 tokamak simulator demonstrate that the policy excels at precisely and stably tracking reference trajectories for key shape parameters across a range of plasma scenarios. This work presents a viable pathway toward developing highly flexible and data-efficient intelligent control systems for future fusion reactors.