IEnSF: Iterative Ensemble Score Filter for Reducing Error in Posterior Score Estimation in Nonlinear Data Assimilation

TL;DR

This paper introduces an iterative ensemble score filter (IEnSF) that reduces errors in posterior score estimation for nonlinear data assimilation, improving accuracy in high-dimensional dynamical systems.

Contribution

The paper develops an iterative algorithm around the reverse-time SDE solver to enhance posterior score estimation in nonlinear settings, addressing structural errors in prior methods.

Findings

IEnSF significantly reduces posterior score estimation error.

Improves accuracy in high-dimensional nonlinear data assimilation.

Demonstrated effectiveness on complex dynamical systems.

Abstract

The Ensemble Score Filter (EnSF) has emerged as a promising approach to leverage score-based diffusion models for solving high-dimensional and nonlinear data assimilation problems. While initial applications of EnSF to the Lorenz-96 model and the quasi-geostrophic system showed potential, the current method employs a heuristic weighted sum to combine the prior and the likelihood score functions. This introduces a structural error into the estimation of the posterior score function in the nonlinear setting. This work addresses this challenge by developing an iterative ensemble score filter (IEnSF) that applies an iterative algorithm as an outer loop around the reverse-time stochastic differential equation solver. When the state dynamics or the observation operator is nonlinear, the iterative algorithm can gradually reduce the posterior score estimation error by improving the accuracy of approximating the conditional expectation of the likelihood score function. The number of iterations required depends on the distance between the prior and posterior distributions and the nonlinearity of the observation operator. Numerical experiments demonstrate that the IEnSF algorithm substantially reduces the error in posterior score estimation in the nonlinear setting and thus improves the accuracy of tracking high-dimensional dynamical systems.