Semi-Implicit Neural Ordinary Differential Equations

TL;DR

This paper introduces a semi-implicit neural ODE method that improves stability and efficiency for stiff problems, outperforming existing approaches in graph learning and dynamical systems.

Contribution

It presents a novel semi-implicit neural ODE framework that leverages the structure of dynamics for better stability and computational efficiency.

Findings

Outperforms existing methods in graph classification

Enables training of complex neural ODEs that are intractable with other methods

Provides computational advantages through efficient linear solves

Abstract

Classical neural ODEs trained with explicit methods are intrinsically limited by stability, crippling their efficiency and robustness for stiff learning problems that are common in graph learning and scientific machine learning. We present a semi-implicit neural ODE approach that exploits the partitionable structure of the underlying dynamics. Our technique leads to an implicit neural network with significant computational advantages over existing approaches because of enhanced stability and efficient linear solves during time integration. We show that our approach outperforms existing approaches on a variety of applications including graph classification and learning complex dynamical systems. We also demonstrate that our approach can train challenging neural ODEs where both explicit methods and fully implicit methods are intractable.