Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods

TL;DR

This paper introduces an implicit single-step method for neural ODEs, enabling them to effectively learn stiff differential equations, which standard methods struggle with, thus broadening their applicability in scientific domains.

Contribution

The paper proposes an implicit neural ODE approach using single-step schemes to handle stiffness, overcoming a major limitation of existing neural ODE methods.

Findings

Implicit neural ODEs successfully learn stiff dynamics.

The method improves stability and accuracy for stiff systems.

Enables neural ODEs to be used in more scientific applications.

Abstract

Stiff systems of ordinary differential equations (ODEs) are pervasive in many science and engineering fields, yet standard neural ODE approaches struggle to learn them. This limitation is the main barrier to the widespread adoption of neural ODEs. In this paper, we propose an approach based on single-step implicit schemes to enable neural ODEs to handle stiffness and demonstrate that our implicit neural ODE method can learn stiff dynamics. This work addresses a key limitation in current neural ODE methods, paving the way for their use in a wider range of scientific problems.