Neural Laplace: Learning diverse classes of differential equations in the Laplace domain

TL;DR

Neural Laplace introduces a framework that models various differential equations in the Laplace domain, effectively handling systems with long-range dependencies, discontinuities, and stiffness, outperforming traditional neural ODEs.

Contribution

It proposes a novel approach to learn diverse classes of differential equations in the Laplace domain, addressing limitations of neural ODEs in modeling complex, discontinuous, and stiff systems.

Findings

Superior performance in modeling diverse DEs

Effective handling of systems with discontinuities

Improved extrapolation of system trajectories

Abstract

Neural Ordinary Differential Equations model dynamical systems with ODEs learned by neural networks. However, ODEs are fundamentally inadequate to model systems with long-range dependencies or discontinuities, which are common in engineering and biological systems. Broader classes of differential equations (DE) have been proposed as remedies, including delay differential equations and integro-differential equations. Furthermore, Neural ODE suffers from numerical instability when modelling stiff ODEs and ODEs with piecewise forcing functions. In this work, we propose Neural Laplace, a unified framework for learning diverse classes of DEs including all the aforementioned ones. Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials. To make learning more efficient, we use the geometrical stereographic map of a Riemann sphere to induce more smoothness in the Laplace domain. In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs, including the ones with complex history dependency and abrupt changes.