On The Temporal Domain of Differential Equation Inspired Graph Neural Networks

TL;DR

This paper introduces TDE-GNN, a neural extension to differential equation-inspired GNNs, capable of modeling complex temporal dynamics beyond traditional first or second-order dependencies, improving performance on graph benchmarks.

Contribution

The paper proposes TDE-GNN, a novel neural extension that learns flexible temporal dependencies in DE-GNNs, surpassing fixed-order temporal models.

Findings

TDE-GNN captures a wider range of temporal dynamics.

Learning temporal dependencies improves benchmark performance.

The model outperforms existing fixed-order DE-GNNs.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable success in modeling complex relationships in graph-structured data. A recent innovation in this field is the family of Differential Equation-Inspired Graph Neural Networks (DE-GNNs), which leverage principles from continuous dynamical systems to model information flow on graphs with built-in properties such as feature smoothing or preservation. However, existing DE-GNNs rely on first or second-order temporal dependencies. In this paper, we propose a neural extension to those pre-defined temporal dependencies. We show that our model, called TDE-GNN, can capture a wide range of temporal dynamics that go beyond typical first or second-order methods, and provide use cases where existing temporal models are challenged. We demonstrate the benefit of learning the temporal dependencies using our method rather than using pre-defined temporal dynamics on several graph benchmarks.