A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines

TL;DR

This paper presents a deep reinforcement learning method for controlling liquid rocket engines during transient phases, outperforming traditional open-loop and PID controllers in adaptability and computational efficiency.

Contribution

It introduces a novel RL-based control approach for the engine start-up phase, demonstrating superior performance and adaptability over existing control strategies.

Findings

RL controller reaches different steady-state points

RL adapts to changing system parameters

RL outperforms open-loop and PID controls

Abstract

Nowadays, liquid rocket engines use closed-loop control at most near steady operating conditions. The control of the transient phases is traditionally performed in open-loop due to highly nonlinear system dynamics. This situation is unsatisfactory, in particular for reusable engines. The open-loop control system cannot provide optimal engine performance due to external disturbances or the degeneration of engine components over time. In this paper, we study a deep reinforcement learning approach for optimal control of a generic gas-generator engine's continuous start-up phase. It is shown that the learned policy can reach different steady-state operating points and convincingly adapt to changing system parameters. A quantitative comparison with carefully tuned open-loop sequences and PID controllers is included. The deep reinforcement learning controller achieves the highest performance and requires only minimal computational effort to calculate the control action, which is a big advantage over approaches that require online optimization, such as model predictive control. control.