Online identification and control of PDEs via Reinforcement Learning methods

TL;DR

This paper presents a reinforcement learning-based method for real-time identification and control of unknown PDE systems using Bayesian linear regression and state-dependent Riccati control, demonstrating convergence through numerical experiments.

Contribution

It introduces a novel online algorithm combining PDE system identification with control using RL, Riccati, and Bayesian methods, enabling adaptive control of unknown PDEs.

Findings

The method converges for infinite horizon control problems.

Numerical experiments validate the effectiveness of the approach.

The approach enables real-time control and identification of unknown PDEs.

Abstract

We focus on the control of unknown Partial Differential Equations (PDEs). The system dynamics is unknown, but we assume we are able to observe its evolution for a given control input, as typical in a Reinforcement Learning framework. We propose an algorithm based on the idea to control and identify on the fly the unknown system configuration. In this work, the control is based on the State-Dependent Riccati approach, whereas the identification of the model on Bayesian linear regression. At each iteration, based on the observed data, we obtain an estimate of the a-priori unknown parameter configuration of the PDE and then we compute the control of the correspondent model. We show by numerical evidence the convergence of the method for infinite horizon control problems.