A statistical learning strategy for closed-loop control of fluid flows

TL;DR

This paper presents a data-efficient, model-free closed-loop control approach for complex fluid systems using statistical learning and reinforcement learning, demonstrated on Lorenz and cylinder flow examples.

Contribution

It introduces a novel control strategy that constructs a Markov model from streaming data without prior system knowledge, enabling robust experimental control.

Findings

Effective control of Lorenz system transitions

Successful drag reduction in cylinder flow

Method requires no prior system modeling

Abstract

This work discusses a closed-loop control strategy for complex systems utilizing scarce and streaming data. A discrete embedding space is first built using hash functions applied to the sensor measurements from which a Markov process model is derived, approximating the complex system's dynamics. A control strategy is then learned using reinforcement learning once rewards relevant with respect to the control objective are identified. This method is designed for experimental configurations, requiring no computations nor prior knowledge of the system, and enjoys intrinsic robustness. It is illustrated on two systems: the control of the transitions of a Lorenz 63 dynamical system, and the control of the drag of a cylinder flow. The method is shown to perform well.