Observation Error Covariance Specification in Dynamical Systems for Data assimilation using Recurrent Neural Networks

TL;DR

This paper introduces a data-driven LSTM-based method for accurately and efficiently estimating observation error covariance matrices in data assimilation for dynamical systems, outperforming traditional methods.

Contribution

It proposes a novel LSTM neural network approach to learn covariance matrices directly from data, eliminating the need for prior assumptions and improving accuracy and efficiency.

Findings

Outperforms classical covariance tuning algorithms in accuracy.

Reduces computational cost compared to traditional methods.

Effective in both Lorenz and shallow water system experiments.

Abstract

Data assimilation techniques are widely used to predict complex dynamical systems with uncertainties, based on time-series observation data. Error covariance matrices modelling is an important element in data assimilation algorithms which can considerably impact the forecasting accuracy. The estimation of these covariances, which usually relies on empirical assumptions and physical constraints, is often imprecise and computationally expensive especially for systems of large dimension. In this work, we propose a data-driven approach based on long short term memory (LSTM) recurrent neural networks (RNN) to improve both the accuracy and the efficiency of observation covariance specification in data assimilation for dynamical systems. Learning the covariance matrix from observed/simulated time-series data, the proposed approach does not require any knowledge or assumption about prior error distribution, unlike classical posterior tuning methods. We have compared the novel approach with two state-of-the-art covariance tuning algorithms, namely DI01 and D05, first in a Lorenz dynamical system and then in a 2D shallow water twin experiments framework with different covariance parameterization using ensemble assimilation. This novel method shows significant advantages in observation covariance specification, assimilation accuracy and computational efficiency.