Deep Learning Based Simulators for the Phosphorus Removal Process Control in Wastewater Treatment via Deep Reinforcement Learning Algorithms

TL;DR

This paper develops high-accuracy models to simulate phosphorus removal in wastewater treatment, enabling deep reinforcement learning algorithms to optimize process control despite some limitations in long-term prediction accuracy.

Contribution

It introduces a novel approach to creating simulation environments for DRL in wastewater treatment using data-driven models without complex system modeling.

Findings

Models achieved over 97% accuracy in identifying the process.

Simulation performance was limited by uncertainty and prediction errors.

The approach reduces reliance on detailed system modeling.

Abstract

Phosphorus removal is vital in wastewater treatment to reduce reliance on limited resources. Deep reinforcement learning (DRL) is a machine learning technique that can optimize complex and nonlinear systems, including the processes in wastewater treatment plants, by learning control policies through trial and error. However, applying DRL to chemical and biological processes is challenging due to the need for accurate simulators. This study trained six models to identify the phosphorus removal process and used them to create a simulator for the DRL environment. Although the models achieved high accuracy (>97%), uncertainty and incorrect prediction behavior limited their performance as simulators over longer horizons. Compounding errors in the models' predictions were identified as one of the causes of this problem. This approach for improving process control involves creating simulation environments for DRL algorithms, using data from supervisory control and data acquisition (SCADA) systems with a sufficient historical horizon without complex system modeling or parameter estimation.