Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemble

TL;DR

This paper demonstrates that deep learning can effectively perform data assimilation for chaotic systems by identifying key instabilities without using ensembles, achieving near-ensemble accuracy with a single forecast state.

Contribution

It introduces a neural network-based analysis scheme that matches ensemble Kalman filter performance without requiring ensemble propagation, revealing insights into chaos dynamics.

Findings

Deep learning analysis approaches match ensemble Kalman filter accuracy.

The method identifies key dynamical perturbations from a single forecast state.

Analysis scheme learns properties related to the ergodic theorem in chaotic systems.

Abstract

We investigate the ability to discover data assimilation (DA) schemes meant for chaotic dynamics with deep learning. The focus is on learning the analysis step of sequential DA, from state trajectories and their observations, using a simple residual convolutional neural network, while assuming the dynamics to be known. Experiments are performed with the Lorenz 96 dynamics, which display spatiotemporal chaos and for which solid benchmarks for DA performance exist. The accuracy of the states obtained from the learned analysis approaches that of the best possibly tuned ensemble Kalman filter, and is far better than that of variational DA alternatives. Critically, this can be achieved while propagating even just a single state in the forecast step. We investigate the reason for achieving ensemble filtering accuracy without an ensemble. We diagnose that the analysis scheme actually identifies key dynamical perturbations, mildly aligned with the unstable subspace, from the forecast state alone, without any ensemble-based covariances representation. This reveals that the analysis scheme has learned some multiplicative ergodic theorem associated to the DA process seen as a non-autonomous random dynamical system.