Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B\'enard convection

TL;DR

This paper introduces a multi-agent reinforcement learning approach to control three-dimensional Rayleigh-Bénard convection, effectively reducing convection intensity and transforming flow patterns, outperforming traditional proportional control methods.

Contribution

First implementation of MARL for 3D Rayleigh-Bénard convection control, demonstrating improved flow regulation and transferability of learned policies across domains.

Findings

Achieved 23.5% reduction in convection at Ra=500

Reduced convection by 8.7% at Ra=750

MARL outperforms proportional control in flow regulation

Abstract

Deep reinforcement learning (DRL) has found application in numerous use-cases pertaining to flow control. Multi-agent RL (MARL), a variant of DRL, has shown to be more effective than single-agent RL in controlling flows exhibiting locality and translational invariance. We present, for the first time, an implementation of MARL-based control of three-dimensional Rayleigh-B\'enard convection (RBC). Control is executed by modifying the temperature distribution along the bottom wall divided into multiple control segments, each of which acts as an independent agent. Two regimes of RBC are considered at Rayleigh numbers $\mathrm{Ra}=500$ and $750$. Evaluation of the learned control policy reveals a reduction in convection intensity by $23.5\%$ and $8.7\%$ at $\mathrm{Ra}=500$ and $750$, respectively. The MARL controller converts irregularly shaped convective patterns to regular straight rolls with lower convection that resemble flow in a relatively more stable regime. We draw comparisons with proportional control at both $\mathrm{Ra}$ and show that MARL is able to outperform the proportional controller. The learned control strategy is complex, featuring different non-linear segment-wise actuator delays and actuation magnitudes. We also perform successful evaluations on a larger domain than used for training, demonstrating that the invariant property of MARL allows direct transfer of the learnt policy.