Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification

TL;DR

This paper enhances Temporal Graph Networks by adding source-target identification to improve their ability to predict future node interactions, outperforming existing models on benchmark datasets.

Contribution

Introducing TGNv2, a modified TGN with source-target identification, enabling it to represent persistent forecasts and autoregressive models, thus improving dynamic node affinity prediction.

Findings

TGNv2 outperforms TGN and other models on benchmark datasets.

Heuristics like moving averages outperform standard TGN.

Source-target identification is essential for modeling persistent forecasts.

Abstract

Despite the successful application of Temporal Graph Networks (TGNs) for tasks such as dynamic node classification and link prediction, they still perform poorly on the task of dynamic node affinity prediction -- where the goal is to predict 'how much' two nodes will interact in the future. In fact, simple heuristic approaches such as persistent forecasts and moving averages over ground-truth labels significantly and consistently outperform TGNs. Building on this observation, we find that computing heuristics over messages is an equally competitive approach, outperforming TGN and all current temporal graph (TG) models on dynamic node affinity prediction. In this paper, we prove that no formulation of TGN can represent persistent forecasting or moving averages over messages, and propose to enhance the expressivity of TGNs by adding source-target identification to each interaction event message. We show that this modification is required to represent persistent forecasting, moving averages, and the broader class of autoregressive models over messages. Our proposed method, TGNv2, significantly outperforms TGN and all current TG models on all Temporal Graph Benchmark (TGB) dynamic node affinity prediction datasets.