Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality

TL;DR

This paper presents a method to enforce fixed points in Neural-ODEs, ensuring they can approximate any velocity field while maintaining their expressive power, with proven universality under fixed-point constraints.

Contribution

The authors introduce a technique to explicitly incorporate fixed points into Neural-ODEs without reducing their expressive capacity, supported by rigorous universality proofs.

Findings

The method allows exact fixed points in Neural-ODEs.

Universality of Neural-ODEs is proven under fixed-point constraints.

Validated on two physical models demonstrating effectiveness.

Abstract

We introduce a technique that enables Neural-ODEs to approximate arbitrary velocity fields with a priori planted fixed-points. Specifically, a recipe is given to explicitly accommodate for a finite collection of points in the reference multi-dimensional space of the Neural-ODE where the velocity field is exactly equal to zero. In this way, the gradient-based training is rigorously constrained inside the prescribed hypothesis class while leaving the expressive power of the Neural-ODE unaltered. We rigorously prove the universality of the Neural-ODE under any local constraints in the velocity field and give a computationally convenient way of imposing the fixed points. Our method is then tested on two paradigmatic physical models.