Vectorized Adjoint Sensitivity Method for Graph Convolutional Neural Ordinary Differential Equations

TL;DR

This paper presents a vectorized implementation of the adjoint sensitivity method for Graph Convolutional Neural ODEs, enabling efficient hardware mapping where autograd is unavailable.

Contribution

It derives a vectorized form of adjoint dynamics for GCDE, facilitating hardware-efficient gradient computation without autograd.

Findings

Provides a derivation of vectorized adjoint dynamics for GCDE

Enables efficient hardware implementation in edge computing and memristor systems

Facilitates gradient calculation without autograd functions

Abstract

This document, as the title stated, is meant to provide a vectorized implementation of adjoint dynamics calculation for Graph Convolutional Neural Ordinary Differential Equations (GCDE). The adjoint sensitivity method is the gradient approximation method for neural ODEs that replaces the back propagation. When implemented on libraries such as PyTorch or Tensorflow, the adjoint can be calculated by autograd functions without the need for a hand-derived formula. In applications such as edge computing and in memristor crossbars, however, autograds are not available, and therefore we need a vectorized derivation of adjoint dynamics to efficiently map the system on hardware. This document will go over the basics, then move on to derive the vectorized adjoint dynamics for GCDE.