Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning

TL;DR

This paper introduces SMARL, an online reinforcement learning approach for developing subgrid-scale closures in geophysical turbulence simulations, effectively capturing extreme events with fewer degrees of freedom and demonstrating good generalization.

Contribution

The paper presents SMARL, a novel reinforcement learning method that develops stable, efficient closures for turbulence simulations, accurately capturing extremes and reducing computational complexity.

Findings

SMARL closures reproduce high-fidelity simulation statistics.

SMARL captures extreme events effectively.

Closures generalize to different flow conditions.

Abstract

Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical. Traditional closures have structural errors, e.g., producing excessive diffusion that dampens extremes. Artificial intelligence has gained attention for closure modeling, but the prediction of extreme events remains challenging. Supervised offline learning needs abundant high-fidelity training data and can lead to instabilities. Online learning algorithms are emerging as an alternative, but reliance on differentiable numerical solvers or scalable optimizers hinders broad use. Here, we introduce SMARL to develop closures for canonical prototypes of atmospheric/oceanic turbulence, using only the enstrophy spectrum, estimated from a few high-fidelity samples, as reward. This reward ensures that the model captures the cascades of scales in these simulations. These online-learned closures enable stable simulations, with up to five orders of magnitude fewer degrees of freedom, that reproduce high-fidelity simulation statistics and capture in particular extremes. We interpret the closures by analyzing the SMARL policy and demonstrate generalization to other flows. The results highlight SMARL as a potent tool for developing closures capable of capturing extremes in atmospheric/oceanic flows, opening new capabilities for effective climate modeling.