Huracan: A skillful end-to-end data-driven system for ensemble data assimilation and weather prediction

TL;DR

Huracan is a novel end-to-end data-driven weather forecasting system that combines ensemble data assimilation with observation inputs, achieving accuracy comparable to top NWP models while using less data.

Contribution

It is the first system to provide ensemble initial conditions and end-to-end forecasts solely from observations, matching ECMWF ENS performance with fewer data.

Findings

Huracan matches or exceeds ECMWF ENS in 75.4% of variable and lead time combinations.

It is the first to deliver ensemble initial conditions and forecasts from observations alone.

Huracan achieves high forecast accuracy with less observational data.

Abstract

Over the past few years, machine learning-based data-driven weather prediction has been transforming operational weather forecasting by providing more accurate forecasts while using a mere fraction of computing power compared to traditional numerical weather prediction (NWP). However, those models still rely on initial conditions from NWP, putting an upper limit on their forecast abilities. A few end-to-end systems have since been proposed, but they have yet to match the forecast skill of state-of-the-art NWP competitors. In this work, we propose Huracan, an observation-driven weather forecasting system which combines an ensemble data assimilation model with a forecast model to produce highly accurate forecasts relying only on observations as inputs. Huracan is not only the first to provide ensemble initial conditions and end-to-end ensemble weather forecasts, but also the first end-to-end system to achieve an accuracy comparable with that of ECMWF ENS, the state-of-the-art NWP competitor, despite using a smaller amount of available observation data. Notably, Huracan matches or exceeds the continuous ranked probability score of ECMWF ENS on 75.4% of the variable and lead time combinations. Our work is a major step forward in end-to-end data-driven weather prediction and opens up opportunities for further improving and revolutionizing operational weather forecasting.