Comparing seven variants of the Ensemble Kalman Filter: How many synthetic experiments are needed?

TL;DR

This study evaluates how many synthetic experiments are necessary to reliably compare the performance of seven Ensemble Kalman Filter variants across different ensemble sizes and model setups.

Contribution

It provides a systematic analysis of the number of synthetic experiments needed for robust performance comparison of EnKF variants in geoscience applications.

Findings

10 synthetic experiments suffice to distinguish RMSE differences >10%

100 experiments needed to detect RMSE differences <2% for certain ensemble sizes

EnKF variant rankings depend on model setup and ensemble size

Abstract

The Ensemble Kalman Filter (EnKF) is a popular estimation technique in the geosciences. It is used as a numerical tool for state vector prognosis and parameter estimation. The EnKF can, for example, help to evaluate the geothermal potential of an aquifer. In such applications, the EnKF is often used with small or medium ensemble sizes. It is therefore of interest to characterize the EnKF behavior for these ensemble sizes. For seven ensemble sizes (50, 70, 100, 250, 500, 1000, 2000) and seven EnKF-variants (Damped, Iterative, Local, Hybrid, Dual, Normal Score and Classical EnKF), we computed 1000 synthetic parameter estimation experiments for two set-ups: a 2D tracer transport problem and a 2D flow problem with one injection well. For each model, the only difference among synthetic experiments was the generated set of random permeability fields. The 1000 synthetic experiments allow to calculate the pdf of the RMSE of the characterization of the permeability field. Comparing mean RMSEs for different EnKF-variants, ensemble sizes and flow/transport set-ups suggests that multiple synthetic experiments are needed for a solid performance comparison. In this work, 10 synthetic experiments were needed to correctly distinguish RMSE differences between EnKF-variants smaller than 10%. For detecting RMSE differences smaller than 2%, 100 synthetic experiments were needed for ensemble sizes 50, 70, 100 and 250. The overall ranking of the EnKF-variants is strongly dependent on the physical model set-up and the ensemble size.