Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimization

TL;DR

This paper introduces a differentiable simulation-based end-to-end learning method for Koopman surrogate models, enhancing control performance and constraint handling in economic nonlinear model predictive control of a CSTR.

Contribution

It presents a novel training algorithm that leverages differentiability of mechanistic models, improving surrogate model performance for control tasks over standard reinforcement learning.

Findings

Achieves similar economic performance to benchmarks

Eliminates constraint violations in control tasks

Outperforms existing training algorithms in a CSTR case study

Abstract

Mechanistic dynamic process models may be too computationally expensive to be usable as part of a real-time capable predictive controller. We present a method for end-to-end learning of Koopman surrogate models for optimal performance in a specific control task. In contrast to previous contributions that employ standard reinforcement learning (RL) algorithms, we use a training algorithm that exploits the differentiability of environments based on mechanistic simulation models to aid the policy optimization. We evaluate the performance of our method by comparing it to that of other training algorithms on an existing economic nonlinear model predictive control (eNMPC) case study of a continuous stirred-tank reactor (CSTR) model. Compared to the benchmark methods, our method produces similar economic performance while eliminating constraint violations. Thus, for this case study, our method outperforms the others and offers a promising path toward more performant controllers that employ dynamic surrogate models.