Adaptive Feedforward Gradient Estimation in Neural ODEs

TL;DR

This paper introduces an adaptive feedforward gradient estimation method for Neural ODEs that enhances efficiency and interpretability by removing the need for backpropagation and adjoint methods, with validated practical benefits.

Contribution

It presents a novel gradient estimation technique for Neural ODEs that reduces computational costs and improves interpretability, differing from traditional backpropagation-based methods.

Findings

Reduces computational overhead compared to adjoint methods

Maintains accuracy while improving efficiency

Validated on practical applications with good performance

Abstract

Neural Ordinary Differential Equations (Neural ODEs) represent a significant breakthrough in deep learning, promising to bridge the gap between machine learning and the rich theoretical frameworks developed in various mathematical fields over centuries. In this work, we propose a novel approach that leverages adaptive feedforward gradient estimation to improve the efficiency, consistency, and interpretability of Neural ODEs. Our method eliminates the need for backpropagation and the adjoint method, reducing computational overhead and memory usage while maintaining accuracy. The proposed approach has been validated through practical applications, and showed good performance relative to Neural ODEs state of the art methods.