Regularization of the ensemble Kalman filter using a non-parametric, non-stationary spatial model

TL;DR

This paper introduces a novel regularization method for the ensemble Kalman filter using a non-parametric, non-stationary spatial model, improving covariance estimation in high-dimensional data assimilation tasks.

Contribution

It proposes a new spatial model-based regularization technique for EnKF, incorporating hyperpriors and a neural Bayes approach for enhanced accuracy and efficiency.

Findings

Significantly improved EnKF performance in simulations.

The method is both accurate and computationally efficient.

Outperforms several existing regularization techniques.

Abstract

The sample covariance matrix of a random vector is a good estimate of the true covariance matrix if the sample size is much larger than the length of the vector. In high-dimensional problems, this condition is never met. As a result, in high dimensions the Ensemble Kalman Filter's (EnKF) ensemble does not contain enough information to specify the prior covariance matrix accurately. This necessitates the need for regularization of the analysis (observation update) problem. We propose a regularization technique based on a new spatial model. The model is a constrained version of the general Gaussian process convolution model. The constraints include local stationarity and smoothness of local spectra. We regularize EnKF by postulating that its prior covariances obey the spatial model. Placing a hyperprior distribution on the model parameters and using the likelihood of the prior ensemble data allows for an optimized use of both the ensemble and the hyperprior. The respective estimator is shown to be consistent. Its neural Bayes implementation proved to be both accurate and computationally efficient. In simulation experiments, the new technique led to substantially better EnKF performance than several existing techniques.