Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows

TL;DR

This paper investigates the application of deep reinforcement learning to control wall-bounded turbulent flows, demonstrating potential for drag reduction and flow structure management through a novel integration of DRL with DNS simulations.

Contribution

It introduces a scalable platform combining DRL with DNS for turbulent flow control, showcasing initial success in drag reduction and flow structure enhancement.

Findings

DRL achieves drag reduction comparable to traditional methods.

The platform demonstrates scalable, efficient communication between DRL and DNS.

Proposes strategies to maintain flow coherence and improve control outcomes.

Abstract

The wall cycle in wall-bounded turbulent flows is a complex turbulence regeneration mechanism that remains not fully understood. This study explores the potential of deep reinforcement learning (DRL) for managing the wall regeneration cycle to achieve desired flow dynamics. We integrate the StableBaselines3 DRL libraries with the open-source DNS solver CaNS to create a robust platform for dynamic flow control. The DRL agent interacts with the DNS environment, learning policies that modify wall boundary conditions to optimize objectives such as the reduction of the skin-friction coefficient or the enhancement of certain coherent structures features. Initial experiments demonstrate the capability of DRL to achieve drag-reduction rates comparable with those achieved via traditional methods, though limited to short time periods. We also propose a strategy to enhance the coherence of velocity streaks, assuming that maintaining straight streaks can inhibit instability and further reduce skin friction. The implementation makes use of the message-passing-interface (MPI) wrappers for efficient communication between the Python-based DRL agent and the DNS solver, ensuring scalability on high-performance computing architectures. Our results highlight the promise of DRL in flow control applications and underscore the need for more advanced control laws and objective functions. Future work will focus on optimizing actuation periods and exploring new computational architectures to extend the applicability and the efficiency of DRL in turbulent flow management.