Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding

TL;DR

This paper introduces CorDGT, a novel dynamic graph transformer that uses correlated spatial-temporal positional encoding to effectively model high-order proximity in continuous-time dynamic graphs, improving representation learning.

Contribution

It proposes a parameter-free, personalized interaction intensity estimation method and a scalable dynamic graph transformer for better modeling of CTDGs.

Findings

Superior performance on multiple datasets

Effective preservation of high-order proximity

Scalability to large-scale graphs

Abstract

Learning effective representations for Continuous-Time Dynamic Graphs (CTDGs) has garnered significant research interest, largely due to its powerful capabilities in modeling complex interactions between nodes. A fundamental and crucial requirement for representation learning in CTDGs is the appropriate estimation and preservation of proximity. However, due to the sparse and evolving characteristics of CTDGs, the spatial-temporal properties inherent in high-order proximity remain largely unexplored. Despite its importance, this property presents significant challenges due to the computationally intensive nature of personalized interaction intensity estimation and the dynamic attributes of CTDGs. To this end, we propose a novel Correlated Spatial-Temporal Positional encoding that incorporates a parameter-free personalized interaction intensity estimation under the weak assumption of the Poisson Point Process. Building on this, we introduce the Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding (CorDGT), which efficiently retains the evolving spatial-temporal high-order proximity for effective node representation learning in CTDGs. Extensive experiments on seven small and two large-scale datasets demonstrate the superior performance and scalability of the proposed CorDGT. The code is available at: https://github.com/wangz3066/CorDGT.