Stochastic parameterization identification using ensemble Kalman filtering combined with expectation-maximization and Newton-Raphson maximum likelihood methods

TL;DR

This paper introduces novel statistical learning methods combining ensemble Kalman filtering with EM and Newton-Raphson algorithms to accurately identify stochastic parameterizations in noisy, coarse-grained geophysical models, overcoming limitations of existing filters.

Contribution

It develops and demonstrates two new methods, EnKF-EM and EnKF-NR, for stochastic parameterization identification using a Bayesian approach, improving accuracy over traditional sequential estimation techniques.

Findings

Successfully identified stochastic parameters in Lorenz-96 models

Achieved good accuracy under moderate observational noise

Proved methods are promising for high-dimensional geophysical models

Abstract

For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal succesfully because both suffer from the collapse of the parameter posterior distribution. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of two methodologies: the maximization of the likelihood via Expectation-Maximization (EM) and Newton-Raphson (NR) algorithms which are mainly applied in the statistic and machine learning communities, and the ensemble Kalman filter (EnKF). The methods are derived using a Bayesian approach for a hidden Markov model. They are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof-of-concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorpored to represent the small-scale dynamics. The algorithms are able to identify an optimal stochastic parameterization with a good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.