Benchmarking AI-based data assimilation to advance data-driven global weather forecasting

TL;DR

This paper introduces DABench, a comprehensive benchmark for evaluating AI-based data assimilation methods in global weather forecasting, demonstrating its effectiveness in improving initial conditions and forecast accuracy.

Contribution

The paper presents DABench, a novel benchmark integrating real-world data for fair comparison of AI-based data assimilation methods in weather forecasting.

Findings

AI-based DA performs competitively with state-of-the-art variational frameworks.

DABench enables objective evaluation of long-term DA cycles.

AI-based DA improves initial conditions for medium-range weather forecasts.

Abstract

Research on Artificial Intelligence (AI)-based Data Assimilation (DA) is expanding rapidly. However, the absence of an objective, comprehensive, and real-world benchmark hinders the fair comparison of diverse methods. Here, we introduce DABench, a benchmark designed for contributing to the development and evaluation of AI-based DA methods. By integrating real-world observations, DABench provides an objective and fair platform for validating long-term closed-loop DA cycles, supporting both deterministic and ensemble configurations. Furthermore, we assess the efficacy of AI-based DA in generating initial conditions for the advanced AI-based weather forecasting model to produce accurate medium-range global weather forecasting. Our dual-validation, utilizing both reanalysis data and independent radiosonde observations, demonstrates that AI-based DA achieves performance competitive with state-of-the-art AI-driven four-dimensional variational frameworks across both global weather DA and medium-range forecasting metrics. We invite the research community to utilize DABench to accelerate the advancement of AI-based DA for global weather forecasting.