ANODEV2: A Coupled Neural ODE Evolution Framework
Tianjun Zhang, Zhewei Yao, Amir Gholami, Kurt Keutzer and, Joseph Gonzalez, George Biros, Michael Mahoney

TL;DR
ANODEV2 introduces a coupled neural ODE framework that jointly evolves network parameters and states, leading to improved training and accuracy over existing Neural ODE methods.
Contribution
It extends Neural ODEs by enabling coupled evolution of parameters and states, providing a more general and trainable framework.
Findings
Achieves higher accuracy than baseline models
Demonstrates trainability of coupled ODE framework
Effective on CIFAR-10 dataset
Abstract
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is an extension of this approach that also allows evolution of the neural network parameters, in a coupled ODE-based formulation. The Neural ODE method introduced earlier is in fact a special case of this new more general framework. We present the formulation of ANODEV2, derive optimality conditions, and implement a coupled reaction-diffusion-advection version of this framework in PyTorch. We present empirical results using several different configurations of ANODEV2, testing them on multiple models on CIFAR-10. We report results showing that this coupled ODE-based framework…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Fluid Dynamics and Turbulent Flows · Nanofluid Flow and Heat Transfer
