Data-driven Methods for Delay Differential Equations

TL;DR

This paper explores data-driven techniques, including extended SINDy algorithms and neural networks, for modeling delay differential equations, demonstrating their effectiveness on classical delay systems and comparing their performance.

Contribution

It introduces novel extensions of the SINDy algorithm for DDEs and neural network-based neural delay differential equations, with MATLAB implementations and comparative analysis.

Findings

SINDy extensions effectively recover DDE dynamics.

Neural networks with trainable delays model DDEs accurately.

Comparative analysis highlights strengths of each approach.

Abstract

Data-driven methodologies are nowadays ubiquitous. Their rapid development and spread have led to applications even beyond the traditional fields of science. As far as dynamical systems and differential equations are concerned, neural networks and sparse identification tools have emerged as powerful approaches to recover the governing equations from available temporal data series. In this chapter we first illustrate possible extensions of the sparse identification of nonlinear dynamics (SINDy) algorithm, originally developed for ordinary differential equations (ODEs), to delay differential equations (DDEs) with discrete, possibly multiple and unknown delays. Two methods are presented for SINDy, one directly tackles the underlying DDE and the other acts on the system of ODEs approximating the DDE through pseudospectral collocation. We also introduce another way of capturing the dynamics of DDEs using neural networks and trainable delays in continuous time, and present the training algorithms developed for these neural delay differential equations (NDDEs). The relevant MATLAB implementations for both the SINDy approach and for the NDDE approach are provided. These approaches are tested on several examples, including classical systems such as the delay logistic and the Mackey-Glass equation, and directly compared to each other on the delayed R\"ossler system. We provide insights on the connection between the approaches and future directions on developing data-driven methods for time delay systems.