Towards Better Evaluation for Dynamic Link Prediction

TL;DR

This paper proposes new evaluation procedures, challenging negative sampling strategies, and introduces six diverse datasets for dynamic link prediction, highlighting the importance of realistic testing and robust methods in evolving graphs.

Contribution

It introduces more stringent evaluation methods, a memorization baseline called EdgeBank, and six new dynamic graph datasets for better assessment of link prediction models.

Findings

EdgeBank performs strongly due to easy negative edges.

Challenging negative sampling improves robustness.

New datasets offer diverse real-world evaluation scenarios.

Abstract

Despite the prevalence of recent success in learning from static graphs, learning from time-evolving graphs remains an open challenge. In this work, we design new, more stringent evaluation procedures for link prediction specific to dynamic graphs, which reflect real-world considerations, to better compare the strengths and weaknesses of methods. First, we create two visualization techniques to understand the reoccurring patterns of edges over time and show that many edges reoccur at later time steps. Based on this observation, we propose a pure memorization baseline called EdgeBank. EdgeBank achieves surprisingly strong performance across multiple settings because easy negative edges are often used in the current evaluation setting. To evaluate against more difficult negative edges, we introduce two more challenging negative sampling strategies that improve robustness and better match real-world applications. Lastly, we introduce six new dynamic graph datasets from a diverse set of domains missing from current benchmarks, providing new challenges and opportunities for future research. Our code repository is accessible at https://github.com/fpour/DGB.git.