Optimizing pulsed blowing parameters for active separation control in a one-sided diffuser using reinforcement learning

TL;DR

This paper demonstrates that reinforcement learning can efficiently optimize pulsed blowing parameters for active flow separation control in a turbulent diffuser, achieving high control authority with minimal episodes.

Contribution

It introduces a reinforcement learning approach using Proximal Policy Optimization to optimize pulsed blowing in turbulent flow separation control, showing high sample efficiency and practical applicability.

Findings

Less than 100 episodes needed for optimal parameters

Low duty cycle actuation is most efficient

Reinforcement learning effectively optimizes turbulent flow control

Abstract

Reinforcement learning is employed to optimize the periodic forcing signal of a pulsed blowing system that controls flow separation in a fully-turbulent $Re_\theta = 1000$ diffuser flow. Based on the state of the wind tunnel experiment that is determined with wall shear-stress measurements, Proximal Policy Optimization is used to iteratively adjust the forcing signal. Out of the reward functions investigated in this study, the incremental reduction of flow reversal per action is shown to be the most sample efficient. Less than 100 episodes are required to find the parameter combination that ensures the highest control authority for a fixed mass flow consumption. Fully consistent with recent studies, the algorithm suggests that the mass flow is used most efficiently when the actuation signal is characterized by a low duty cycle where the pulse duration is small compared to the pulsation period. The results presented in this paper promote the application of reinforcement learning for optimization tasks based on turbulent, experimental data.