Neural Flows: Efficient Alternative to Neural ODEs

TL;DR

Neural Flows offer a computationally efficient alternative to Neural ODEs by directly modeling solution curves with neural networks, maintaining modeling power while reducing computational costs.

Contribution

The paper introduces neural flows as a new approach that models ODE solution curves directly, eliminating the need for numerical solvers and providing versatile architectures for various applications.

Findings

Achieves computational efficiency comparable to or better than Neural ODEs.

Demonstrates strong generalization in time series forecasting and density estimation.

Provides theoretical conditions for valid flow architectures.

Abstract

Neural ordinary differential equations describe how values change in time. This is the reason why they gained importance in modeling sequential data, especially when the observations are made at irregular intervals. In this paper we propose an alternative by directly modeling the solution curves - the flow of an ODE - with a neural network. This immediately eliminates the need for expensive numerical solvers while still maintaining the modeling capability of neural ODEs. We propose several flow architectures suitable for different applications by establishing precise conditions on when a function defines a valid flow. Apart from computational efficiency, we also provide empirical evidence of favorable generalization performance via applications in time series modeling, forecasting, and density estimation.