Inaccuracy of Ensemble-Based Covariance Propagation, Beyond Sampling Error

TL;DR

This paper reveals that ensemble-based covariance propagation in data assimilation can be highly inaccurate when correlation lengths are near grid resolution, due to fundamental discrepancies with continuum covariance dynamics, beyond sampling errors.

Contribution

It demonstrates through experiments and analysis that ensemble covariance propagation can be fundamentally inaccurate, challenging assumptions in current data assimilation practices.

Findings

Ensemble covariances can be significantly inaccurate near grid scale.

Errors in covariance propagation are beyond sampling errors and cannot be fixed by standard methods.

Discrete covariance dynamics can differ fundamentally from continuum covariance dynamics.

Abstract

Modern data assimilation schemes typically use the same discrete dynamical model to evolve the state estimate in time also to approximate the evolution, or propagation, of the estimation error covariance. Ensemble-based methods, such as the ensemble Kalman filter, approximate the evolution of the covariance through the propagation of individual ensemble members. Thus, it is tacitly assumed that if the discrete state propagation and resulting mean state estimates are accurate, then the ensemble-based discrete covariance propagation will be accurate as well, apart from sampling errors due to limited ensemble size. Through a series of numerical experiments supported by analytical results, we demonstrate that this assumption is false when correlation length scales approach grid resolution. We show for states that satisfy advective dynamics, that while the discrete state propagation and ensemble mean state estimates are accurate, the corresponding ensemble covariances can be remarkably inaccurate, well beyond that expected from sampling errors or typical numerical discretization errors. The underlying problem is a fundamental discrepancy between discrete covariance propagation and the continuum covariance dynamics, which we can identify because the exact continuum covariance dynamics are known. Errors in the ensemble covariances, which can be at least one order of magnitude larger than those of the mean state when correlation lengths begin to approach grid scale, cannot be rectified by the usual methods, such as covariance inflation and localization. This work brings to light a fundamental problem for data assimilation schemes that propagate covariances using the same discrete dynamical model used to propagate the state.