From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems

TL;DR

This paper introduces Fourier NODEs, a simulation-free neural ODE framework that uses Fourier analysis to directly match target vector fields, improving training efficiency, accuracy, and robustness in modeling complex systems.

Contribution

We propose Fourier NODEs, a novel framework that leverages Fourier analysis for direct vector field matching, reducing computational costs and enhancing robustness in neural ODE training.

Findings

Outperforms state-of-the-art methods in training time

Achieves more accurate dynamics prediction

Demonstrates robustness across complex systems

Abstract

Modeling complex systems using standard neural ordinary differential equations (NODEs) often faces some essential challenges, including high computational costs and susceptibility to local optima. To address these challenges, we propose a simulation-free framework, called Fourier NODEs (FNODEs), that effectively trains NODEs by directly matching the target vector field based on Fourier analysis. Specifically, we employ the Fourier analysis to estimate temporal and potential high-order spatial gradients from noisy observational data. We then incorporate the estimated spatial gradients as additional inputs to a neural network. Furthermore, we utilize the estimated temporal gradient as the optimization objective for the output of the neural network. Later, the trained neural network generates more data points through an ODE solver without participating in the computational graph, facilitating more accurate estimations of gradients based on Fourier analysis. These two steps form a positive feedback loop, enabling accurate dynamics modeling in our framework. Consequently, our approach outperforms state-of-the-art methods in terms of training time, dynamics prediction, and robustness. Finally, we demonstrate the superior performance of our framework using a number of representative complex systems.