Enhanced Reinforcement Learning-based Process Synthesis via Quantum Computing

TL;DR

This paper introduces quantum reinforcement learning algorithms for process synthesis, demonstrating improved scalability and efficiency over classical methods in moderate-sized problems.

Contribution

It develops a generalized quantum RL framework with state encoding algorithms that decouple qubit requirements from problem size, enhancing scalability.

Findings

Quantum approaches perform competitively on small problems.

Quantum algorithms show improved efficiency per parameter on moderate problems.

All methods can identify optimal designs in small search spaces.

Abstract

In this work, we present quantum reinforcement learning (RL) as a solution strategy for process synthesis problems. Building on our prior work, we develop a generalized framework that formally poses process synthesis as a Markov decision process and introduces quantum-enhanced RL algorithms to solve it with improved scalability. Earlier implementations of quantum-based RL for process synthesis were limited by qubit requirements, which scaled poorly with problem complexity. This work overcomes this challenge by introducing state encoding algorithms to decouple qubit requirements from problem size. A classical RL-based solution strategy is used as a baseline to benchmark the quantum algorithms under identical training conditions. All algorithms are evaluated across a flowsheet synthesis problem of increasing unit counts to analyze their performance and scalability. Results show that all approaches are capable of identifying the optimal flowsheet designs in small design spaces. For moderate-scale unit counts, quantum approaches demonstrate competitive performance on a per-episode basis and improved efficiency on a per-parameter basis versus the classical RL benchmark. This work provides a foundation for future quantum computing applications within process systems engineering, establishes a controlled benchmark for comparing classical and quantum algorithms, and shows that the proposed quantum variants remain competitive for the process synthesis problem examined in this work.