Temporal Graph Pattern Machine

TL;DR

The paper introduces the Temporal Graph Pattern Machine (TGPM), a novel framework that learns generalized evolving patterns in dynamic networks using a Transformer backbone and self-supervised pre-training, outperforming existing methods in link prediction tasks.

Contribution

TGPM shifts focus to directly learning transferable temporal evolution patterns with a Transformer-based model and self-supervised tasks, overcoming limitations of prior task-centric approaches.

Findings

Achieves state-of-the-art results in link prediction

Demonstrates strong cross-domain transferability

Effectively models long-range temporal dependencies

Abstract

Temporal graph learning is pivotal for deciphering dynamic systems, where the core challenge lies in explicitly modeling the underlying evolving patterns that govern network transformation. However, prevailing methods are predominantly task-centric and rely on restrictive assumptions -- such as short-term dependency modeling, static neighborhood semantics, and retrospective time usage. These constraints hinder the discovery of transferable temporal evolution mechanisms. To address this, we propose the Temporal Graph Pattern Machine (TGPM), a foundation framework that shifts the focus toward directly learning generalized evolving patterns. TGPM conceptualizes each interaction as an interaction patch synthesized via temporally-biased random walks, thereby capturing multi-scale structural semantics and long-range dependencies that extend beyond immediate neighborhoods. These patches are processed by a Transformer-based backbone designed to capture global temporal regularities while adapting to context-specific interaction dynamics. To further empower the model, we introduce a suite of self-supervised pre-training tasks -- specifically masked token modeling and next-time prediction -- to explicitly encode the fundamental laws of network evolution. Extensive experiments show that TGPM consistently achieves state-of-the-art performance in both transductive and inductive link prediction, demonstrating exceptional cross-domain transferability. Our code has been released in https://github.com/antman9914/TGPM.