Stabilizing Rayleigh-Benard convection with reinforcement learning trained on a reduced-order model

TL;DR

This paper introduces a novel control framework combining data-driven manifold dynamics with reinforcement learning to stabilize Rayleigh-Benard convection, achieving significant heat transfer reduction in high-dimensional turbulent flows.

Contribution

It develops a reduced-order model using POD and autoencoders, enabling efficient RL training and effective flow stabilization strategies.

Findings

Achieved 16-23% reduction in Nusselt number.

Controlled flow exhibits suppressed fluctuations and steady heat flux.

Method is scalable and physically interpretable.

Abstract

Rayleigh-Benard convection (RBC) is a canonical system for buoyancy-driven turbulence and heat transport, central to geophysical and industrial flows. Developing efficient control strategies remains challenging at high Rayleigh numbers, where fully resolved simulations are computationally expensive. We use a control framework that couples data-driven manifold dynamics (DManD) with reinforcement learning (RL) to suppress convective heat transfer. We find a coordinate transformation to a low-dimensional system using POD and autoencoders, and then learn an evolution equation for this low-dimensional state using neural ODEs. The reduced model reproduces key system features while enabling rapid policy training. Policies trained in the DManD environment and deployed in DNS achieve a 16-23 % reduction in the Nusselt number for both single- and dual-boundary actuation. Physically, the learned strategy modulates near-wall heat flux to stabilize and thicken the thermal boundary layer, weaken plume ejection, and damp the wall-driven instabilities that seed convective bursts. Crucially, the controller drives the flow toward a quasi-steady state characterized by suppressed temporal fluctuations and spatially steady heat-flux patterns. This work establishes DManD-RL as a physically interpretable, scalable approach for turbulence control in high-dimensional flows.