Automated Synthesis of Steady-State Continuous Processes using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning approach for automated steady-state flowsheet synthesis in process engineering, using a novel game-based exploration method called SynGameZero, demonstrated on a reaction-distillation system.

Contribution

It presents a new RL-based framework with a game-theoretic exploration strategy for flowsheet synthesis without relying on heuristics or prior knowledge.

Findings

Successfully applied to a reaction-distillation process

Demonstrates effective exploration in complex process design

Outperforms traditional heuristic-based methods

Abstract

Automated flowsheet synthesis is an important field in computer-aided process engineering. The present work demonstrates how reinforcement learning can be used for automated flowsheet synthesis without any heuristics of prior knowledge of conceptual design. The environment consists of a steady-state flowsheet simulator that contains all physical knowledge. An agent is trained to take discrete actions and sequentially built up flowsheets that solve a given process problem. A novel method named SynGameZero is developed to ensure good exploration schemes in the complex problem. Therein, flowsheet synthesis is modelled as a game of two competing players. The agent plays this game against itself during training and consists of an artificial neural network and a tree search for forward planning. The method is applied successfully to a reaction-distillation process in a quaternary system.