Controlling overestimation of error covariance in ensemble Kalman filters with sparse observations: A variance limiting Kalman filter

TL;DR

This paper introduces a variance limiting Kalman filter (VLKF) that effectively manages overestimation of error covariance in ensemble Kalman filters with sparse observations, improving stability and skill in data assimilation.

Contribution

The paper develops a novel VLKF method that incorporates known variances of unobservable variables to enhance ensemble Kalman filter performance with partial data.

Findings

VLKF improves data assimilation stability.

Incorporating variance info enhances forecast skill.

Method effective in Lorenz-96 system simulations.

Abstract

We consider the problem of an ensemble Kalman filter when only partial observations are available. In particular we consider the situation where the observational space consists of variables which are directly observable with known observational error, and of variables of which only their climatic variance and mean are given. To limit the variance of the latter poorly resolved variables we derive a variance limiting Kalman filter (VLKF) in a variational setting. We analyze the variance limiting Kalman filter for a simple linear toy model and determine its range of optimal performance. We explore the variance limiting Kalman filter in an ensemble transform setting for the Lorenz-96 system, and show that incorporating the information of the variance of some un-observable variables can improve the skill and also increase the stability of the data assimilation procedure.