Neural Integro-Differential Equations

TL;DR

Neural IDEs are a new deep learning framework that models complex non-local dynamical systems, including non-Markovian brain dynamics, by learning integral operators, enabling better prediction and interpretability.

Contribution

We introduce Neural IDEs, a novel neural network-based framework for modeling non-local and non-Markovian dynamics using integro-differential equations.

Findings

NIDE outperforms existing models on toy and brain activity datasets.

NIDE can decompose dynamics into Markovian and non-Markovian parts.

The learned integral operator provides insights into underlying neural dynamics.

Abstract

Modeling continuous dynamical systems from discretely sampled observations is a fundamental problem in data science. Often, such dynamics are the result of non-local processes that present an integral over time. As such, these systems are modeled with Integro-Differential Equations (IDEs); generalizations of differential equations that comprise both an integral and a differential component. For example, brain dynamics are not accurately modeled by differential equations since their behavior is non-Markovian, i.e. dynamics are in part dictated by history. Here, we introduce the Neural IDE (NIDE), a novel deep learning framework based on the theory of IDEs where integral operators are learned using neural networks. We test NIDE on several toy and brain activity datasets and demonstrate that NIDE outperforms other models. These tasks include time extrapolation as well as predicting dynamics from unseen initial conditions, which we test on whole-cortex activity recordings in freely behaving mice. Further, we show that NIDE can decompose dynamics into their Markovian and non-Markovian constituents via the learned integral operator, which we test on fMRI brain activity recordings of people on ketamine. Finally, the integrand of the integral operator provides a latent space that gives insight into the underlying dynamics, which we demonstrate on wide-field brain imaging recordings. Altogether, NIDE is a novel approach that enables modeling of complex non-local dynamics with neural networks.