Assessing the ability of a stretched-grid deep-learning weather prediction model to capture physical balances

TL;DR

This study evaluates a deep learning weather prediction model's ability to accurately simulate a severe storm, highlighting its strengths and limitations in capturing physical balances and mesoscale features.

Contribution

It provides a detailed comparison of a regional DLWP model's physical realism against operational NWP during an extreme weather event.

Findings

Bris model performs well in RMSE but struggles with mesoscale features.

Bris significantly disrupts atmospheric balances due to fine-scale noise.

Unrealistic spatial gradients in Bris output affect physical consistency.

Abstract

Weather forecasting has traditionally relied on Numerical Weather Prediction (NWP) models, which simulate weather by solving the governing fluid equations. Recently, the emergence of Deep Learning Weather Prediction (DLWP) models has opened a new era in weather forecasting, offering a data-driven alternative to classical NWP approaches. Regional DLWP models such as the stretched-grid model Bris developed by Met Norway, have demonstrated performance on par with, or even slightly better than regional NWP models across a range of standard forecast metrics. By overcoming the coarse horizontal resolution that constrained earlier global data-driven models, the operational use of regional DLWP systems now appears increasingly promising. Nevertheless, the performance of such models during extreme events is generally inferior to that of regional NWP models, and comprehensive evaluations of their ability to generate physically realistic forecasts are still lacking. Here, we present a study comparing the physical consistency of the deterministic version of Bris with the control run of the operational MetCoOp Ensemble Prediction System (MEPS) in forecasting the severe extratropical cyclone Poly, which hit the Netherlands on 5 July 2023. We examine whether Bris accurately represents deviations from key atmospheric balances and whether it reproduces expected dynamics of the storm. We show that, despite its relatively good performance in terms of RMSE, Bris struggles to capture important mesoscale features of the event and that it significantly disrupts several atmospheric balances. This unrealistic disruption is mainly linked to the fine-scale noise evidenced in its output fields, which leads to incorrect and unrealistic spatial gradients. These results raise critical questions for improving AI-based models to better represent extreme events and how to ensure physical consistency in their predictions.