Inductive Representation Learning on Temporal Graphs

TL;DR

This paper introduces TGAT, a novel attention-based model for inductive learning on temporal graphs that captures dynamic node features and topological changes over time, improving performance on classification and prediction tasks.

Contribution

The paper proposes TGAT, a new temporal graph attention layer utilizing self-attention and a novel time encoding, enabling inductive learning on evolving graphs.

Findings

TGAT outperforms state-of-the-art baselines on benchmark datasets.

The model effectively captures temporal and topological dynamics.

It generalizes well to both transductive and inductive tasks.

Abstract

Inductive representation learning on temporal graphs is an important step toward salable machine learning on real-world dynamic networks. The evolving nature of temporal dynamic graphs requires handling new nodes as well as capturing temporal patterns. The node embeddings, which are now functions of time, should represent both the static node features and the evolving topological structures. Moreover, node and topological features can be temporal as well, whose patterns the node embeddings should also capture. We propose the temporal graph attention (TGAT) layer to efficiently aggregate temporal-topological neighborhood features as well as to learn the time-feature interactions. For TGAT, we use the self-attention mechanism as building block and develop a novel functional time encoding technique based on the classical Bochner's theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves. The proposed approach handles both node classification and link prediction task, and can be naturally extended to include the temporal edge features. We evaluate our method with transductive and inductive tasks under temporal settings with two benchmark and one industrial dataset. Our TGAT model compares favorably to state-of-the-art baselines as well as the previous temporal graph embedding approaches.