DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning

TL;DR

DeepThermal is a novel AI system that uses offline reinforcement learning to optimize combustion efficiency in thermal power units, combining data-driven simulation and real data for effective control.

Contribution

The paper introduces a new offline RL framework, MORE, and a combustion process simulator, enabling effective optimization of TPGU combustion control using historical data.

Findings

DeepThermal improves combustion efficiency in real power plants.

MORE outperforms existing offline RL algorithms on benchmark tasks.

The system demonstrates successful deployment in four large coal-fired power plants.

Abstract

Optimizing the combustion efficiency of a thermal power generating unit (TPGU) is a highly challenging and critical task in the energy industry. We develop a new data-driven AI system, namely DeepThermal, to optimize the combustion control strategy for TPGUs. At its core, is a new model-based offline reinforcement learning (RL) framework, called MORE, which leverages historical operational data of a TGPU to solve a highly complex constrained Markov decision process problem via purely offline training. In DeepThermal, we first learn a data-driven combustion process simulator from the offline dataset. The RL agent of MORE is then trained by combining real historical data as well as carefully filtered and processed simulation data through a novel restrictive exploration scheme. DeepThermal has been successfully deployed in four large coal-fired thermal power plants in China. Real-world experiments show that DeepThermal effectively improves the combustion efficiency of TPGUs. We also report the superior performance of MORE by comparing with the state-of-the-art algorithms on the standard offline RL benchmarks.