Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering

TL;DR

This paper provides a comprehensive survey and tutorial on reinforcement learning methods tailored for process systems engineering, highlighting current applications, challenges, and future research directions in this interdisciplinary field.

Contribution

It offers the first detailed tutorial on RL for PSE and synthesizes existing applications, challenges, and emerging trends in integrating RL with process systems engineering.

Findings

RL methods are effectively applied in process control and optimization.

Current challenges include data efficiency and system complexity.

Emerging directions involve hybrid models and real-time learning.

Abstract

Sequential decision making under uncertainty is central to many Process Systems Engineering (PSE) challenges, where traditional methods often face limitations related to controlling and optimizing complex and stochastic systems. Reinforcement Learning (RL) offers a data-driven approach to derive control policies for such challenges. This paper presents a survey and tutorial on RL methods, tailored for the PSE community. We deliver a tutorial on RL, covering fundamental concepts and key algorithmic families including value-based, policy-based and actor-critic methods. Subsequently, we survey existing applications of these RL techniques across various PSE domains, such as in fed-batch and continuous process control, process optimization, and supply chains. We conclude with PSE focused discussion of specialized techniques and emerging directions. By synthesizing the current state of RL algorithm development and implications for PSE this work identifies successes, challenges, trends, and outlines avenues for future research at the interface of these fields.