Reduced-Order Autodifferentiable Ensemble Kalman Filters

TL;DR

This paper presents ROAD-EnKFs, a framework that learns low-dimensional surrogate models for complex dynamical systems to improve state reconstruction and forecasting efficiency and accuracy.

Contribution

It introduces a novel reduced-order autodifferentiable ensemble Kalman filter that learns latent dynamics and decoders for efficient state estimation.

Findings

Higher accuracy for low-dimensional structured dynamics

Lower computational cost compared to existing methods

Comparable accuracy when structure is absent

Abstract

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system. Our reduced-order autodifferentiable ensemble Kalman filters (ROAD-EnKFs) learn a latent low-dimensional surrogate model for the dynamics and a decoder that maps from the latent space to the state space. The learned dynamics and decoder are then used within an ensemble Kalman filter to reconstruct and forecast the state. Numerical experiments show that if the state dynamics exhibit a hidden low-dimensional structure, ROAD-EnKFs achieve higher accuracy at lower computational cost compared to existing methods. If such structure is not expressed in the latent state dynamics, ROAD-EnKFs achieve similar accuracy at lower cost, making them a promising approach for surrogate state reconstruction and forecasting.