Data assimilation for chaotic dynamics

TL;DR

This paper reviews recent advances in understanding how chaos affects data assimilation methods, highlighting the importance of ensemble size, the role of unstable modes, and the challenges in high-dimensional chaotic systems.

Contribution

It introduces new insights into the functioning of ensemble Kalman filters in chaotic systems and tests the limits of particle filters in high-dimensional chaos, emphasizing the importance of unstable subspaces.

Findings

Ensemble Kalman filter requires sufficient ensemble members to track near-null modes.

Discarding observations in low-growth directions does not mitigate the curse of dimensionality in particle filters.

The rank of the unstable-neutral subspace determines the number of particles needed.

Abstract

Chaos is ubiquitous in physical systems. The associated sensitivity to initial conditions is a significant obstacle in forecasting the weather and other geophysical fluid flows. Data assimilation is the process whereby the uncertainty in initial conditions is reduced by the astute combination of model predictions and real-time data. This chapter reviews recent findings from investigations on the impact of chaos on data assimilation methods: for the Kalman filter and smoother in linear systems, analytic results are derived; for their ensemble-based versions and nonlinear dynamics, numerical results provide insights. The focus is on characterising the asymptotic statistics of the Bayesian posterior in terms of the dynamical instabilities, differentiating between deterministic and stochastic dynamics. We also present two novel results. Firstly, we study the functioning of the ensemble Kalman filter in the context of a chaotic, coupled, atmosphere-ocean model with a quasi-degenerate spectrum of Lyapunov exponents, showing the importance of having sufficient ensemble members to track all of the near-null modes. Secondly, for the fully non-Gaussian method of the particle filter, numerical experiments are conducted to test whether the curse of dimensionality can be mitigated by discarding observations in the directions of little dynamical growth of uncertainty. The results refute this option, most likely because the particles already embody this information on the chaotic system. The results also suggest that it is the rank of the unstable-neutral subspace of the dynamics, and not that of the observation operator, that determines the required number of particles. We finally discuss how knowledge of the random attractor can play a role in the development of future data assimilation schemes for chaotic multiscale systems with large scale separation.