Attention on flow control: transformer-based reinforcement learning for lift regulation in highly disturbed flows

TL;DR

This paper introduces a transformer-based reinforcement learning framework for aerodynamic lift regulation in highly disturbed flows, outperforming traditional control methods and generalizing well to long gust sequences.

Contribution

It develops a novel transformer-based RL approach for lift control in turbulent flows, incorporating pretraining and transfer learning for improved performance and generalization.

Findings

Transformer-based RL outperforms proportional control.

Control strategy generalizes to arbitrarily long gust sequences.

Quarter-chord pitching achieves better lift regulation with less effort.

Abstract

A linear flow control strategy designed for weak disturbances may not remain effective in sequences of strong disturbances due to nonlinear interactions, but it is sensible to leverage it for developing a better strategy. In the present study, we propose a transformer-based reinforcement learning (RL) framework to learn an effective control strategy for regulating aerodynamic lift in arbitrarily long gust sequences via pitch control. The random gusts produce intermittent, high-variance flows observed only through limited surface pressure sensors, making this control problem inherently challenging compared to stationary flows. The transformer addresses the challenge of partial observability from the limited surface pressures. We demonstrate that the training can be accelerated with two techniques -- pretraining with an expert policy (here, linear control) and task-level transfer learning (here, extending a policy trained on isolated gusts to multiple gusts). We show that the learned strategy outperforms the best proportional control, with the performance gap widening as the number of gusts increases. The control strategy learned in an environment with a small number of successive gusts is shown to effectively generalize to an environment with an arbitrarily long sequence of gusts. We investigate the pivot configuration and show that quarter-chord pitching control can achieve superior lift regulation with substantially less control effort compared to mid-chord pitching control. Through a decomposition of the lift, we attribute this advantage to the dominant added-mass contribution accessible via quarter-chord pitching.