Integrating process design and control using reinforcement learning

TL;DR

This paper introduces a novel approach combining reinforcement learning with process design optimization, enabling efficient simultaneous design and control of engineering systems, outperforming existing strategies.

Contribution

It proposes embedding reinforcement learning-based control into the design process to decouple and efficiently solve the bilevel optimization problem.

Findings

Outperforms current state-of-the-art methods in case studies

Demonstrates effective decoupling of design and control optimization

Validates approach in two practical case studies

Abstract

To create efficient-high performing processes, one must find an optimal design with its corresponding controller that ensures optimal operation in the presence of uncertainty. When comparing different process designs, for the comparison to be meaningful, each design must involve its optimal operation. Therefore, to optimize a process' design, one must address design and control simultaneously. For this, one can formulate a bilevel optimization problem, with the design as the outer problem in the form of a mixed-integer nonlinear program (MINLP) and a stochastic optimal control as the inner problem. This is intractable by most approaches. In this paper we propose to compute the optimal control using reinforcement learning, and then embed this controller into the design problem. This allows to decouple the solution procedure, while having the same optimal result as if solving the bilevel problem. The approach is tested in two case studies and the performance of the controller is evaluated. The case studies indicate that the proposed approach outperforms current state-of-the-art simultaneous design and control strategies. This opens a new avenue to address simultaneous design and control of engineering systems.