Learning Chaotic Systems and Long-Term Predictions with Neural Jump ODEs

TL;DR

This paper introduces an advanced neural jump ODE model for predicting complex stochastic and deterministic systems, demonstrating improved long-term accuracy especially in chaotic systems like a double pendulum.

Contribution

The paper presents novel enhancements to the Path-dependent Neural Jump ODE, enabling it to learn long-term dynamics of chaotic and stochastic systems more accurately.

Findings

Enhanced model closely matches true chaotic dynamics.

Standard model prediction diverges after half the evaluation time.

Proposed improvements enable reliable long-term predictions for stochastic datasets.

Abstract

The Path-dependent Neural Jump ODE (PD-NJ-ODE) is a model for online prediction of generic (possibly non-Markovian) stochastic processes with irregular (in time) and potentially incomplete (with respect to coordinates) observations. It is a model for which convergence to the $L^2$-optimal predictor, which is given by the conditional expectation, is established theoretically. Thereby, the training of the model is solely based on a dataset of realizations of the underlying stochastic process, without the need of knowledge of the law of the process. In the case where the underlying process is deterministic, the conditional expectation coincides with the process itself. Therefore, this framework can equivalently be used to learn the dynamics of ODE or PDE systems solely from realizations of the dynamical system with different initial conditions. We showcase the potential of our method by applying it to the chaotic system of a double pendulum. When training the standard PD-NJ-ODE method, we see that the prediction starts to diverge from the true path after about half of the evaluation time. In this work we enhance the model with two novel ideas, which independently of each other improve the performance of our modelling setup. The resulting dynamics match the true dynamics of the chaotic system very closely. The same enhancements can be used to provably enable the PD-NJ-ODE to learn long-term predictions for general stochastic datasets, where the standard model fails. This is verified in several experiments.