Implicit degree bias in the link prediction task

TL;DR

This paper reveals that the standard link prediction benchmark is biased towards high-degree nodes, and proposes a degree-corrected evaluation method to improve the assessment and training of graph models.

Contribution

It identifies an implicit degree bias in link prediction benchmarks and introduces a degree-corrected task for fairer evaluation and better model training.

Findings

Null degree-based method performs nearly optimally under standard bias.

Degree correction reduces overfitting to node degrees.

Improved model training with the degree-corrected benchmark.

Abstract

Link prediction -- a task of distinguishing actual hidden edges from random unconnected node pairs -- is one of the quintessential tasks in graph machine learning. Despite being widely accepted as a universal benchmark and a downstream task for representation learning, the validity of the link prediction benchmark itself has been rarely questioned. Here, we show that the common edge sampling procedure in the link prediction task has an implicit bias toward high-degree nodes and produces a highly skewed evaluation that favors methods overly dependent on node degree, to the extent that a ``null'' link prediction method based solely on node degree can yield nearly optimal performance. We propose a degree-corrected link prediction task that offers a more reasonable assessment that aligns better with the performance in the recommendation task. Finally, we demonstrate that the degree-corrected benchmark can more effectively train graph machine-learning models by reducing overfitting to node degrees and facilitating the learning of relevant structures in graphs.