Universal Approximation of Dynamical Systems by Semi-Autonomous Neural ODEs and Applications

TL;DR

This paper introduces semi-autonomous neural ODEs (SA-NODEs), demonstrating their universal approximation capabilities for dynamical systems with fewer parameters, backed by theoretical proofs and numerical experiments showing improved performance over vanilla NODEs.

Contribution

The paper presents SA-NODEs, a novel variant of neural ODEs with fewer parameters, and establishes their universal approximation properties both theoretically and numerically.

Findings

SA-NODEs can approximate dynamical systems with vanishing error as parameters increase.

SA-NODEs outperform vanilla NODEs in capturing system dynamics.

Numerical results confirm the effectiveness and reduced complexity of SA-NODEs.

Abstract

In this paper, we introduce semi-autonomous neural ordinary differential equations (SA-NODEs), a variation of the vanilla NODEs, employing fewer parameters. We investigate the universal approximation properties of SA-NODEs for dynamical systems from both a theoretical and a numerical perspective. Within the assumption of a finite-time horizon, under general hypotheses we establish an asymptotic approximation result, demonstrating that the error vanishes as the number of parameters goes to infinity. Under additional regularity assumptions, we further specify this convergence rate in relation to the number of parameters, utilizing quantitative approximation results in the Barron space. Based on the previous result, we prove an approximation rate for transport equations by their neural counterparts. Our numerical experiments validate the effectiveness of SA-NODEs in capturing the dynamics of various ODE systems and transport equations. Additionally, we compare SA-NODEs with vanilla NODEs, highlighting the superior performance and reduced complexity of our approach.