State Space Models on Temporal Graphs: A First-Principles Study

TL;DR

This paper introduces GraphSSM, a novel state space model tailored for temporal graphs, integrating structural information to effectively model dynamic systems and outperform existing methods on multiple benchmarks.

Contribution

It extends state space models to temporal graphs by incorporating structural regularization, creating a new framework called GraphSSM for dynamic graph modeling.

Findings

GraphSSM outperforms existing models on various benchmarks.

Structural regularization improves temporal graph modeling accuracy.

The continuous-time formulation addresses long-range dependency issues.

Abstract

Over the past few years, research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. In practice, temporal graphs are formalized as an ordered sequence of static graph snapshots observed at discrete time points. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. Yet, despite the promising results, RNNs struggle with long-range dependencies, while transformers are burdened by quadratic computational complexity. Recently, state space models (SSMs), which are framed as discretized representations of an underlying continuous-time linear dynamical system, have garnered substantial attention and achieved breakthrough advancements in independent sequence modeling. In this work, we undertake a principled investigation that extends SSM theory to temporal graphs by integrating structural information into the online approximation objective via the adoption of a Laplacian regularization term. The emergent continuous-time system introduces novel algorithmic challenges, thereby necessitating our development of GraphSSM, a graph state space model for modeling the dynamics of temporal graphs. Extensive experimental results demonstrate the effectiveness of our GraphSSM framework across various temporal graph benchmarks.