Machine Learning-Based Covariance Correction for Ensemble Kalman Filter with Limited Ensemble Size

TL;DR

This paper introduces a machine learning-enhanced ensemble Kalman filter that improves accuracy with small ensembles by predicting covariance differences, demonstrated on Lorenz systems.

Contribution

The study develops a novel ML-based covariance correction method that enhances EnKF performance with limited ensemble sizes, balancing accuracy and computational cost.

Findings

The proposed method outperforms standard EnKF with the same ensemble size.

Numerical experiments show significant accuracy improvements on Lorenz systems.

The approach maintains computational efficiency while improving analysis accuracy.

Abstract

Data assimilation (DA) integrates numerical model forecasts with observations to achieve the optimal state estimation. Ensemble-based methods, such as the ensemble Kalman filter (EnKF), are widely used for state estimation for high-dimensional and nonlinear dynamic systems. However, their performance strongly depends on the ensemble size, therefore causing a tradeoff problem between analysis accuracy and computational cost. To address this problem, this study presents a machine learning-based EnKF framework that maintains high accuracy with a relatively small ensemble size. Specifically, a multilayer perceptron (MLP) function is built to predict the difference between the forecast error covariances estimated from a limited ensemble and a sufficiently large ensemble, with the latter being assumed to be an accurate approximation of the underlying truth. This predicted covariance difference term is then incorporated into the EnKF algorithm via an element-wise scaling strategy, resulting in an amended forecast covariance matrix that better approximates the true uncertainty level and sequentially produces more accurate analysis results. To demonstrate the feasibility and robustness of the proposed algorithm, we perform a set of numerical experiments with the Lorenz-63 and Lorenz-96 systems under various configurations, and the results consistently indicate that the proposed algorithm can significantly outperform the standard EnKF with the same limited ensemble size, by achieving notably higher analysis accuracy while remaining computationally efficient. This approach provides a practical and feasible pathway to accurate and computationally efficient data assimilation for high-dimensional and nonlinear dynamic systems.