Uncertainty quantification for data-driven weather models

TL;DR

This paper explores methods to quantify uncertainty in data-driven weather forecasts, enhancing the reliability of AI-based models like Pangu-Weather by providing probabilistic predictions and comparing different approaches.

Contribution

It systematically compares ensemble and statistical methods for uncertainty quantification in data-driven weather models, demonstrating improved probabilistic forecasts over traditional physics-based models.

Findings

Probabilistic forecasts outperform deterministic predictions.

Ensemble methods improve uncertainty estimation.

Results show better accuracy up to 5 days lead time.

Abstract

Artificial intelligence (AI)-based data-driven weather forecasting models have experienced rapid progress over the last years. Recent studies, with models trained on reanalysis data, achieve impressive results and demonstrate substantial improvements over state-of-the-art physics-based numerical weather prediction models across a range of variables and evaluation metrics. Beyond improved predictions, the main advantages of data-driven weather models are their substantially lower computational costs and the faster generation of forecasts, once a model has been trained. However, most efforts in data-driven weather forecasting have been limited to deterministic, point-valued predictions, making it impossible to quantify forecast uncertainties, which is crucial in research and for optimal decision making in applications. Our overarching aim is to systematically study and compare uncertainty quantification methods to generate probabilistic weather forecasts from a state-of-the-art deterministic data-driven weather model, Pangu-Weather. Specifically, we compare approaches for quantifying forecast uncertainty based on generating ensemble forecasts via perturbations to the initial conditions, with the use of statistical and machine learning methods for post-hoc uncertainty quantification. In a case study on medium-range forecasts of selected weather variables over Europe, the probabilistic forecasts obtained by using the Pangu-Weather model in concert with uncertainty quantification methods show promising results and provide improvements over ensemble forecasts from the physics-based ensemble weather model of the European Centre for Medium-Range Weather Forecasts for lead times of up to 5 days.