Optimizing Operation Recipes with Reinforcement Learning for Safe and Interpretable Control of Chemical Processes

TL;DR

This paper introduces a reinforcement learning-based method to optimize chemical process operation recipes, improving safety, interpretability, and data efficiency compared to traditional RL and control techniques.

Contribution

It presents a novel approach that uses expert-embedded recipes and RL to optimize parameters, reducing data needs and enhancing safety and interpretability.

Findings

Approaches near optimal controller performance in simulations.

Requires less data than traditional RL methods.

Effectively manages safety and quality constraints.

Abstract

Optimal operation of chemical processes is vital for energy, resource, and cost savings in chemical engineering. The problem of optimal operation can be tackled with reinforcement learning, but traditional reinforcement learning methods face challenges due to hard constraints related to quality and safety that must be strictly satisfied, and the large amount of required training data. Chemical processes often cannot provide sufficient experimental data, and while detailed dynamic models can be an alternative, their complexity makes it computationally intractable to generate the needed data. Optimal control methods, such as model predictive control, also struggle with the complexity of the underlying dynamic models. Consequently, many chemical processes rely on manually defined operation recipes combined with simple linear controllers, leading to suboptimal performance and limited flexibility. In this work, we propose a novel approach that leverages expert knowledge embedded in operation recipes. By using reinforcement learning to optimize the parameters of these recipes and their underlying linear controllers, we achieve an optimized operation recipe. This method requires significantly less data, handles constraints more effectively, and is more interpretable than traditional reinforcement learning methods due to the structured nature of the recipes. We demonstrate the potential of our approach through simulation results of an industrial batch polymerization reactor, showing that it can approach the performance of optimal controllers while addressing the limitations of existing methods.