Learning Dynamic Graph Embeddings with Neural Controlled Differential Equations

TL;DR

This paper introduces GN-CDEs, a continuous-time neural framework for modeling the complex, evolving dynamics of nodes and structures in dynamic graphs, improving representation learning and robustness.

Contribution

It proposes a novel neural controlled differential equation model for dynamic graphs, enabling continuous-time evolution modeling without piecewise integration.

Findings

Effective in capturing complex graph dynamics

Robust to missing data

Outperforms existing methods in dynamic graph tasks

Abstract

This paper focuses on representation learning for dynamic graphs with temporal interactions. A fundamental issue is that both the graph structure and the nodes own their own dynamics, and their blending induces intractable complexity in the temporal evolution over graphs. Drawing inspiration from the recent progress of physical dynamic models in deep neural networks, we propose Graph Neural Controlled Differential Equations (GN-CDEs), a continuous-time framework that jointly models node embeddings and structural dynamics by incorporating a graph enhanced neural network vector field with a time-varying graph path as the control signal. Our framework exhibits several desirable characteristics, including the ability to express dynamics on evolving graphs without piecewise integration, the capability to calibrate trajectories with subsequent data, and robustness to missing observations. Empirical evaluation on a range of dynamic graph representation learning tasks demonstrates the effectiveness of our proposed approach in capturing the complex dynamics of dynamic graphs.