Neural delay differential equations: learning non-Markovian closures for partially known dynamical systems

TL;DR

This paper introduces Neural Delay Differential Equations (NDDEs), a continuous-time model capturing non-Markovian dynamics from partial observations using memory effects with finite time delays, validated on diverse datasets.

Contribution

The paper proposes NDDEs, a novel continuous-time framework that learns non-Markovian dynamics with finite delays, addressing partial observability in dynamical systems.

Findings

NDDEs outperform LSTM and ANODE on various datasets.

NDDEs effectively model memory effects in chaotic and experimental systems.

The approach is data-efficient and suitable for real-world applications.

Abstract

Recent advances in learning dynamical systems from data have shown significant promise. However, many existing methods assume access to the full state of the system -- an assumption that is rarely satisfied in practice, where systems are typically monitored through a limited number of sensors, leading to partial observability. To address this challenge, we draw inspiration from the Mori-Zwanzig formalism, which provides a theoretical connection between hidden variables and memory terms. Motivated by this perspective, we introduce a constant-lag Neural Delay Differential Equations (NDDEs) framework, providing a continuous-time approach for learning non-Markovian dynamics directly from data. These memory effects are captured using a finite set of time delays, which are identified via the adjoint method. We validate the proposed approach on a range of datasets, including synthetic systems, chaotic dynamics, and experimental measurements, such as the Kuramoto-Sivashinsky equation and cavity-flow experiments. Results demonstrate that NDDEs compare favourably with existing approaches for partially observed systems, including long short-term memory (LSTM) networks and augmented neural ordinary differential equations (ANODEs). Overall, NDDEs offer a principled and data-efficient framework for modelling non-Markovian dynamics under partial observability. An open-source implementation accompanies this article.