SmartFlow: A CFD-solver-agnostic deep reinforcement learning framework for computational fluid dynamics on HPC platforms

TL;DR

SmartFlow is a flexible, solver-agnostic deep reinforcement learning framework designed for high-performance computing platforms, enabling advanced fluid dynamics research through seamless integration with various CFD solvers and multi-agent algorithms.

Contribution

It introduces a novel, solver-agnostic DRL framework that integrates with HPC CFD solvers using low-latency communication, facilitating complex fluid dynamics applications.

Findings

Demonstrated drag reduction in cylinder flow using single-agent control.

Showcased multi-agent wake control with GPU-accelerated spectral-element code.

Enabled wall-model learning for large-eddy simulation with finite-difference solver.

Abstract

Deep reinforcement learning (DRL) is emerging as a powerful tool for fluid-dynamics research, encompassing active flow control, autonomous navigation, turbulence modeling and discovery of novel numerical schemes. We introduce SmartFlow, a CFD-solver-agnostic framework for both single- and multi-agent DRL algorithms that can easily integrate with MPI-parallel CPU and GPU-accelerated solvers. Built on Relexi and SmartSOD2D, SmartFlow uses the SmartSim infrastructure library and our newly developed SmartRedis-MPI library to enable asynchronous, low-latency, in-memory communication between CFD solvers and Python-based DRL algorithms. SmartFlow leverages PyTorch's Stable-Baselines3 for training, which provides a modular, Gym-like environment API. We demonstrate its versatility via three case studies: single-agent synthetic-jet control for drag reduction in a cylinder flow simulated by the high-order FLEXI solver, multi-agent cylinder wake control using the GPU-accelerated spectral-element code SOD2D, and multi-agent wall-model learning for large-eddy simulation with the finite-difference solver CaLES. SmartFlow's CFD-solver-agnostic design and seamless HPC integration is promising to accelerate RL-driven fluid-mechanics studies.