Learning to traverse convective flows at moderate to high Rayleigh numbers

TL;DR

This paper demonstrates how reinforcement learning enables a particle to navigate turbulent convective flows efficiently, revealing flow organization impacts and developing interpretable strategies for robust traversal.

Contribution

It introduces a physics-informed RL approach for navigating turbulent convection, linking flow structures with navigation success and energy efficiency.

Findings

Success rate increases with propulsion limit at moderate Rayleigh numbers

Energy required decreases despite longer traversal times at higher Rayleigh numbers

Flow reorganization affects navigation pathways and barrier crossing strategies

Abstract

We study the navigation of a self-propelled inertial particle in two-dimensional Rayleigh--B\'enard convection at Prandtl number $Pr = 0.71$ and cell aspect ratio $\Gamma = 4$ for Rayleigh numbers $Ra$ ranging from $10^{7}$ to $10^{11}$. A reinforcement-learning (RL) controller selects the propulsive acceleration, subject to an upper bound $\mathcal{A}_{\max}$, to achieve a prescribed horizontal displacement. We find that the success rate increases abruptly with $\mathcal{A}_{\max}$ at moderate $Ra$, whereas at higher $Ra$ the transition becomes more gradual and shifts to larger $\mathcal{A}_{\max}$. Moreover, although the completion time increases with $Ra$, the propulsion energy required for successful traversal decreases. Proper orthogonal decomposition (POD) reveals that these performance differences arise from reorganisation of the carrier flow. At moderate $Ra$, the dominant large-scale circulation partitions the domain through robust transport barriers, requiring a finite thrust surplus to cross them; at higher $Ra$, energy is distributed across many modes, the barriers fragment, and transient plume-assisted pathways emerge. Compared with a constant-heading baseline, the learned policy aligns with local currents and consumes significantly less energy. Lagrangian coherent structure (LCS) analysis further shows that the RL agent inherently learns to cross repelling barriers and surf along attracting pathways. Finally, by mapping these behaviours onto the local Eulerian flow topology using Voronoi tessellation and the $Q$-criterion, we distil an interpretable, physics-based heuristic strategy that achieves robust navigability. These results connect turbulent-flow organisation with autonomous navigation under bounded actuation.