Safe Chance Constrained Reinforcement Learning for Batch Process Control

TL;DR

This paper introduces a data-driven safe reinforcement learning approach using Gaussian processes to handle plant-model mismatch and ensure joint chance constraint satisfaction in batch process control.

Contribution

It extends existing safe RL methods by incorporating Gaussian process-based uncertainty to address plant-model mismatch in process control.

Findings

Successfully accounts for process uncertainty and plant-model mismatch

Enables satisfaction of joint chance constraints in case studies

Outperforms traditional nonlinear model predictive control

Abstract

Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independently of explicit assumption of process uncertainty. Recent focus on engineering applications has been directed towards the development of safe RL controllers. Previous works have proposed approaches to account for constraint satisfaction through constraint tightening from the domain of stochastic model predictive control. Here, we extend these approaches to account for plant-model mismatch. Specifically, we propose a data-driven approach that utilizes Gaussian processes for the offline simulation model and use the associated posterior uncertainty prediction to account for joint chance constraints and plant-model mismatch. The method is benchmarked against nonlinear model predictive control via case studies. The results demonstrate the ability of the methodology to account for process uncertainty, enabling satisfaction of joint chance constraints even in the presence of plant-model mismatch.