Towards Better Dynamic Graph Learning: New Architecture and Unified Library

TL;DR

DyGFormer is a novel Transformer-based architecture for dynamic graph learning that effectively captures long-term dependencies using neighbor co-occurrence encoding and patching, supported by a comprehensive library for reproducibility.

Contribution

The paper introduces DyGFormer, a simple yet effective Transformer-based model with a new encoding scheme and patching technique, along with DyGLib, a unified library for dynamic graph research.

Findings

Achieves state-of-the-art results on most datasets.

Effectively models long-term temporal dependencies.

Highlights the importance of standardized evaluation protocols.

Abstract

We propose DyGFormer, a new Transformer-based architecture for dynamic graph learning. DyGFormer is conceptually simple and only needs to learn from nodes' historical first-hop interactions by: (1) a neighbor co-occurrence encoding scheme that explores the correlations of the source node and destination node based on their historical sequences; (2) a patching technique that divides each sequence into multiple patches and feeds them to Transformer, allowing the model to effectively and efficiently benefit from longer histories. We also introduce DyGLib, a unified library with standard training pipelines, extensible coding interfaces, and comprehensive evaluating protocols to promote reproducible, scalable, and credible dynamic graph learning research. By performing exhaustive experiments on thirteen datasets for dynamic link prediction and dynamic node classification tasks, we find that DyGFormer achieves state-of-the-art performance on most of the datasets, demonstrating its effectiveness in capturing nodes' correlations and long-term temporal dependencies. Moreover, some results of baselines are inconsistent with previous reports, which may be caused by their diverse but less rigorous implementations, showing the importance of DyGLib. All the used resources are publicly available at https://github.com/yule-BUAA/DyGLib.