Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at $Re_D=3900$

TL;DR

This paper introduces a multi-agent reinforcement learning approach for active flow control on a turbulent cylinder at Re_D=3900, achieving significant drag reduction and high efficiency compared to classical methods.

Contribution

It presents a novel multi-agent reinforcement learning framework with a multi-stage training approach for active flow control in turbulent regimes, demonstrating superior efficiency and effectiveness.

Findings

Achieved approximately 9% drag reduction.

Demonstrated mass cost efficiency two orders of magnitude lower than classical methods.

Developed a sophisticated cooperative control strategy utilizing a wide bandwidth of frequencies.

Abstract

This study presents novel drag reduction active-flow-control (AFC) strategies} for a three-dimensional cylinder immersed in a flow at a Reynolds number based on freestream velocity and cylinder diameter of $Re_D=3900$. The cylinder in this subcritical flow regime has been extensively studied in the literature and is considered a classic case of turbulent flow arising from a bluff body. The strategies presented are explored through the use of deep reinforcement learning. The cylinder is equipped with 10 independent zero-net-mass-flux jet pairs, distributed on the top and bottom surfaces, which define the AFC setup. The method is based on the coupling between a computational-fluid-dynamics solver and a multi-agent reinforcement-learning (MARL) framework using the proximal-policy-optimization algorithm. This work introduces a multi-stage training approach to expand the exploration space and enhance drag reduction stabilization. By accelerating training through the exploitation of local invariants with MARL, a drag reduction of approximately 9% is achieved. The cooperative closed-loop strategy developed by the agents is sophisticated, as it utilizes a wide bandwidth of mass-flow-rate frequencies, which classical control methods are unable to match. Notably, the mass cost efficiency is demonstrated to be two orders of magnitude lower than that of classical control methods reported in the literature. These developments represent a significant advancement in active flow control in turbulent regimes, critical for industrial applications.