# Reinforcement Learning for Batch Bioprocess Optimization

## TL;DR

This paper introduces a reinforcement learning approach using policy gradients and recurrent neural networks to optimize complex batch bioprocesses, addressing challenges like stochasticity and plant-model mismatch.

## Contribution

It presents a novel RL-based optimization strategy that updates control policies from preliminary models using real plant data, outperforming traditional methods in complex scenarios.

## Key findings

- Effective policy updates from preliminary models to real plant data
- Successful application on three diverse bioprocess case studies
- Advantages over nonlinear model predictive control demonstrated

## Abstract

Bioprocesses have received a lot of attention to produce clean and sustainable alternatives to fossil-based materials. However, they are generally difficult to optimize due to their unsteady-state operation modes and stochastic behaviours. Furthermore, biological systems are highly complex, therefore plant-model mismatch is often present. To address the aforementioned challenges we propose a Reinforcement learning based optimization strategy for batch processes. In this work, we applied the Policy Gradient method from batch-to-batch to update a control policy parametrized by a recurrent neural network. We assume that a preliminary process model is available, which is exploited to obtain a preliminary optimal control policy. Subsequently, this policy is updatedbased on measurements from thetrueplant. The capabilities of our proposed approach were tested on three case studies (one of which is nonsmooth) using a more complex process model for thetruesystemembedded with adequate process disturbance. Lastly, we discussed the advantages and disadvantages of this strategy compared against current existing approaches such as nonlinear model predictive control.