A condensing approach to multiple shooting neural ordinary differential equation

TL;DR

This paper introduces a condensing approach to effectively incorporate equality constraints in multiple-shooting neural ODEs, enhancing stability and training efficiency for complex trajectories.

Contribution

The paper proposes a novel condensing method to incorporate shooting constraints in neural ODEs, enabling stable training with first-order optimizers.

Findings

Improved stability in training neural ODEs with multiple-shooting.

Effective incorporation of equality constraints using condensing.

Enhanced training efficiency with first-order methods.

Abstract

Multiple-shooting is a parameter estimation approach for ordinary differential equations. In this approach, the trajectory is broken into small intervals, each of which can be integrated independently. Equality constraints are then applied to eliminate the shooting gap between the end of the previous trajectory and the start of the next trajectory. Unlike single-shooting, multiple-shooting is more stable, especially for highly oscillatory and long trajectories. In the context of neural ordinary differential equations, multiple-shooting is not widely used due to the challenge of incorporating general equality constraints. In this work, we propose a condensing-based approach to incorporate these shooting equality constraints while training a multiple-shooting neural ordinary differential equation (MS-NODE) using first-order optimization methods such as Adam.