Tensor-Var: Efficient Four-Dimensional Variational Data Assimilation

TL;DR

Tensor-Var introduces a novel framework combining kernel methods and deep learning to linearize nonlinear dynamics in 4D-Var data assimilation, significantly improving accuracy and computational efficiency.

Contribution

It proposes a new linearization approach for 4D-Var using kernel conditional mean embedding, with theoretical guarantees and scalable deep feature learning.

Findings

Outperforms conventional 4D-Var in accuracy

Achieves 10-20 times faster computation

Effective on chaotic systems and weather prediction

Abstract

Variational data assimilation estimates the dynamical system states by minimizing a cost function that fits the numerical models with the observational data. Although four-dimensional variational assimilation (4D-Var) is widely used, it faces high computational costs in complex nonlinear systems and depends on imperfect state-observation mappings. Deep learning (DL) offers more expressive approximators, while integrating DL models into 4D-Var is challenging due to their nonlinearities and lack of theoretical guarantees in assimilation results. In this paper, we propose Tensor-Var, a novel framework that integrates kernel conditional mean embedding (CME) with 4D-Var to linearize nonlinear dynamics, achieving convex optimization in a learned feature space. Moreover, our method provides a new perspective for solving 4D-Var in a linear way, offering theoretical guarantees of consistent assimilation results between the original and feature spaces. To handle large-scale problems, we propose a method to learn deep features using neural networks within the Tensor-Var framework. Experiments on chaotic systems and global weather prediction with real-time observations show that Tensor-Var outperforms conventional and DL hybrid 4D-Var baselines in accuracy while achieving a 10- to 20-fold speed improvement.