End-to-End Reinforcement Learning of Koopman Models for eNMPC of an Air Separation Unit

TL;DR

This paper demonstrates that an end-to-end reinforcement learning approach for Koopman models effectively scales to large, complex air separation units, achieving economic benefits while maintaining constraint satisfaction in eNMPC applications.

Contribution

It extends a previously proposed RL-based Koopman modeling method to large-scale, realistic industrial systems for the first time.

Findings

Method scales well to large-scale systems

Achieves economic performance comparable to system identification methods

Avoids constraint violations in control applications

Abstract

With our recently proposed method based on reinforcement learning (Mayfrank et al. (2024), Comput. Chem. Eng. 190), Koopman surrogate models can be trained for optimal performance in specific (economic) nonlinear model predictive control ((e)NMPC) applications. So far, our method has exclusively been demonstrated on a small-scale case study. Herein, we show that our method scales well to a more challenging demand response case study built on a large-scale model of a single-product (nitrogen) air separation unit. Across all numerical experiments, we assume observability of only a few realistically measurable plant variables. Compared to a purely system identification-based Koopman eNMPC, which generates small economic savings but frequently violates constraints, our method delivers similar economic performance while avoiding constraint violations.