Evaluation of forecasts by a global data-driven weather model with and without probabilistic post-processing at Norwegian stations

TL;DR

This study evaluates the forecast accuracy of a global data-driven weather model, Pangu-Weather, compared to traditional NWP models at Norwegian stations, highlighting the benefits of probabilistic post-processing.

Contribution

It provides a comparative analysis of a recent data-driven model against established NWP models with and without post-processing, demonstrating its competitive performance.

Findings

Pangu-Weather slightly outperforms ECMWF models for temperature.

MEPS ensemble model provides the best forecasts overall.

Post-processing significantly improves forecast quality, especially for global models.

Abstract

During the last two years, tremendous progress in global data-driven weather models trained on numerical weather prediction (NWP) re-analysis data has been made. The most recent models trained on the ERA5 at 0.25{\deg} resolution demonstrate forecast quality on par with ECMWF's high-resolution model with respect to a wide selection of verification metrics. In this study, one of these models, the Pangu-Weather, is compared to several NWP models with and without probabilistic post-processing for 2-meter temperature and 10-meter wind speed forecasting at 183 Norwegian SYNOP stations up to +60 hours ahead. The NWP models included are the ECMWF HRES, ECMWF ENS and the Harmonie-AROME ensemble model MEPS with 2.5 km spatial resolution. Results show that the performances of the global models are on the same level with Pangu-Weather being slightly better than the ECMWF models for temperature and slightly worse for wind speed. The MEPS model clearly provided the best forecasts for both parameters. The post-processing improved the forecast quality considerably for all models, but to a larger extent for the coarse-resolution global models due to stronger systematic deficiencies in these. Apart from this, the main characteristics in the scores were more or less the same with and without post-processing. Our results thus confirm the conclusions from other studies that global data-driven models are promising for operational weather forecasting.