Accurate and Efficient Hybrid-Ensemble Atmospheric Data Assimilation in Latent Space with Uncertainty Quantification

TL;DR

This paper introduces HLOBA, a hybrid-ensemble atmospheric data assimilation method operating in a learned latent space, which achieves high accuracy, efficiency, and uncertainty quantification for weather prediction and climate reanalysis.

Contribution

HLOBA is a novel latent-space DA method that combines autoencoder-based representation with Bayesian updates, improving accuracy and efficiency over traditional methods.

Findings

HLOBA matches the performance of 4D-Var in analysis and forecast skill.

HLOBA provides element-wise uncertainty estimates that identify large-error regions.

The method is flexible and applicable to various forecasting models.

Abstract

Data assimilation (DA) combines model forecasts and observations to estimate the optimal state of the atmosphere with its uncertainty, providing initial conditions for weather prediction and reanalyses for climate research. Yet, existing traditional and machine-learning DA methods struggle to achieve accuracy, efficiency and uncertainty quantification simultaneously. Here, we propose HLOBA (Hybrid-Ensemble Latent Observation-Background Assimilation), a three-dimensional hybrid-ensemble DA method that operates in an atmospheric latent space learned via an autoencoder (AE). HLOBA maps both model forecasts and observations into a shared latent space via the AE encoder and an end-to-end Observation-to-Latent-space mapping network (O2Lnet), respectively, and fuses them through a Bayesian update with weights inferred from time-lagged ensemble forecasts. Both idealized and real-observation experiments demonstrate that HLOBA matches dynamically constrained four-dimensional DA methods in both analysis and forecast skill, while achieving end-to-end inference-level efficiency and theoretical flexibility applies to any forecasting model. Moreover, by exploiting the error decorrelation property of latent variables, HLOBA enables element-wise uncertainty estimates for its latent analysis and propagates them to model space via the decoder. Idealized experiments show that this uncertainty highlights large-error regions and captures their seasonal variability.