Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs

TL;DR

This paper introduces a reinforcement learning controller that integrates a pretrained DeepONet, trained via backstepping, within the soft actor-critic framework to effectively stabilize PDE systems, outperforming traditional methods.

Contribution

It presents a novel method combining backstepping-pretrained DeepONet with SAC for PDE stabilization, enhancing control performance over standard approaches.

Findings

Outperforms standard SAC in PDE stabilization tasks.

Pretrained DeepONet effectively captures backstepping control features.

Successfully stabilizes hyperbolic and reaction-diffusion PDEs.

Abstract

This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the soft actor-critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and serves as a feature extractor, replacing the convolutional neural networks (CNNs) layers in the original actor and critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable ffrst-order hyperbolic PDE and an unstable reactiondiffusion PDE. Simulation results demonstrate that the proposed method outperforms the standard SAC, SAC with an untrained DeepONet, and the backstepping controller on both systems.