Performance of Ensemble Kalman filters in large dimensions

TL;DR

This paper provides a rigorous stochastic analysis of Ensemble Kalman filters' accuracy and covariance fidelity in large-dimensional systems with small ensemble sizes, addressing the practical challenges of bias and model errors.

Contribution

It offers the first theoretical analysis explaining why EnKF performs well in high dimensions with small ensembles, including bias and error control mechanisms.

Findings

EnKF maintains accuracy with small ensembles in large dimensions.

Covariance inflation and localization are crucial for EnKF performance.

The analysis quantifies bias and model errors in high-dimensional settings.

Abstract

Contemporary data assimilation often involves more than a million prediction variables. Ensemble Kalman filters (EnKF) have been developed by geoscientists. They are successful indispensable tools in science and engineering, because they allow for computationally cheap low ensemble state approximation for extremely large dimensional turbulent dynamical systems. The practical finite ensemble filter like EnKF necessarily involve modifications such as covariance inflation and localization, and it is a genuine mystery why they perform so well with small ensemble sizes in large dimensions. This paper provides the first rigorous stochastic analysis of the accuracy and covariance fidelity of EnKF in the practical regime where the ensemble size is much smaller than the large ambient dimension for EnKFs with random coefficients. A challenging issue overcome here is that EnKF in huge dimensions introduces unavoidable bias and model errors which need to be controlled and estimated.