T-GRAB: A Synthetic Diagnostic Benchmark for Learning on Temporal Graphs

TL;DR

T-GRAB introduces synthetic, interpretable tasks to systematically evaluate the temporal reasoning capabilities of graph neural networks, revealing their limitations in capturing core temporal patterns like causality and long-range dependencies.

Contribution

This work presents T-GRAB, a novel benchmark with synthetic tasks designed to assess and diagnose the temporal reasoning skills of TGNNs, highlighting current models' shortcomings.

Findings

Current TGNNs struggle with causal inference tasks.

Models have difficulty capturing long-range dependencies.

Traditional benchmarks may hide these limitations.

Abstract

Dynamic graph learning methods have recently emerged as powerful tools for modelling relational data evolving through time. However, despite extensive benchmarking efforts, it remains unclear whether current Temporal Graph Neural Networks (TGNNs) effectively capture core temporal patterns such as periodicity, cause-and-effect, and long-range dependencies. In this work, we introduce the Temporal Graph Reasoning Benchmark (T-GRAB), a comprehensive set of synthetic tasks designed to systematically probe the capabilities of TGNNs to reason across time. T-GRAB provides controlled, interpretable tasks that isolate key temporal skills: counting/memorizing periodic repetitions, inferring delayed causal effects, and capturing long-range dependencies over both spatial and temporal dimensions. We evaluate 11 temporal graph learning methods on these tasks, revealing fundamental shortcomings in their ability to generalize temporal patterns. Our findings offer actionable insights into the limitations of current models, highlight challenges hidden by traditional real-world benchmarks, and motivate the development of architectures with stronger temporal reasoning abilities. The code for T-GRAB can be found at: https://github.com/alirezadizaji/T-GRAB.