Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

TL;DR

This paper introduces a deep reinforcement learning approach for the online control of complex stochastic partial differential equations, demonstrated on turbulent fluid flow models.

Contribution

It formulates SPDE control as a reinforcement learning problem and applies a distributed deep deterministic policy gradient method for high-dimensional systems.

Findings

Effective control of stochastic Burgers' equation demonstrated

Deep RL approach handles high-dimensional state-action spaces

Potential for real-time control in complex dynamical systems

Abstract

In many areas, such as the physical sciences, life sciences, and finance, control approaches are used to achieve a desired goal in complex dynamical systems governed by differential equations. In this work we formulate the problem of controlling stochastic partial differential equations (SPDE) as a reinforcement learning problem. We present a learning-based, distributed control approach for online control of a system of SPDEs with high dimensional state-action space using deep deterministic policy gradient method. We tested the performance of our method on the problem of controlling the stochastic Burgers' equation, describing a turbulent fluid flow in an infinitely large domain.