Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control

TL;DR

This paper introduces TATD3, a novel reinforcement learning algorithm with twin actor networks and new reward functions, designed to improve control in complex, nonlinear batch processes.

Contribution

It develops the TATD3 algorithm by integrating twin actor networks into TD3 and proposes two novel reward functions for enhanced batch process control.

Findings

TATD3 outperforms existing RL algorithms in batch process control tasks.

The proposed reward functions improve learning efficiency and control accuracy.

TATD3 demonstrates robustness across various process examples.

Abstract

Control of batch processes is a difficult task due to their complex nonlinear dynamics and unsteady-state operating conditions within batch and batch-to-batch. It is expected that some of these challenges can be addressed by developing control strategies that directly interact with the process and learning from experiences. Recent studies in the literature have indicated the advantage of having an ensemble of actors in actor-critic Reinforcement Learning (RL) frameworks for improving the policy. The present study proposes an actor-critic RL algorithm, namely, twin actor twin delayed deep deterministic policy gradient (TATD3), by incorporating twin actor networks in the existing twin-delayed deep deterministic policy gradient (TD3) algorithm for the continuous control. In addition, two types of novel reward functions are also proposed for TATD3 controller. We showcase the efficacy of the TATD3 based controller for various batch process examples by comparing it with some of the existing RL algorithms presented in the literature.