From Link Prediction to Forecasting: Addressing Challenges in Batch-based Temporal Graph Learning

TL;DR

This paper highlights the limitations of batch-based evaluation in dynamic link prediction for temporal graphs and proposes reformulating the task as link forecasting to improve fairness and accuracy.

Contribution

It identifies issues with traditional batch evaluation and introduces a reformulation as link forecasting to better utilize temporal data.

Findings

Batch evaluation causes information loss and unfair comparisons.

Reformulating as link forecasting improves evaluation accuracy.

Traditional methods may produce skewed performance results.

Abstract

Dynamic link prediction is an important problem considered in many recent works that propose approaches for learning temporal edge patterns. To assess their efficacy, models are evaluated on continuous-time and discrete-time temporal graph datasets, typically using a traditional batch-oriented evaluation setup. However, as we show in this work, a batch-oriented evaluation is often unsuitable and can cause several issues. Grouping edges into fixed-sized batches regardless of their occurrence time leads to information loss or leakage, depending on the temporal granularity of the data. Furthermore, fixed-size batches create time windows with different durations, resulting in an inconsistent dynamic link prediction task. In this work, we empirically show how traditional batch-based evaluation leads to skewed model performance and hinders the fair comparison of methods. We mitigate this problem by reformulating dynamic link prediction as a link forecasting task that better accounts for temporal information present in the data.