Evaluating Link Prediction Accuracy on Dynamic Networks with Added and Removed Edges

TL;DR

This paper examines the challenges of evaluating link prediction methods in dynamic networks with both added and removed edges, highlighting issues with current metrics and proposing a unified evaluation approach.

Contribution

It identifies limitations of existing evaluation metrics for dynamic link prediction and introduces a new unified metric for more accurate assessment.

Findings

Current metrics can be misleading in dynamic networks.

Separating evaluation of added and removed edges improves accuracy.

A new unified metric effectively summarizes link prediction performance.

Abstract

The task of predicting future relationships in a social network, known as link prediction, has been studied extensively in the literature. Many link prediction methods have been proposed, ranging from common neighbors to probabilistic models. Recent work by Yang et al. has highlighted several challenges in evaluating link prediction accuracy. In dynamic networks where edges are both added and removed over time, the link prediction problem is more complex and involves predicting both newly added and newly removed edges. This results in new challenges in the evaluation of dynamic link prediction methods, and the recommendations provided by Yang et al. are no longer applicable, because they do not address edge removal. In this paper, we investigate several metrics currently used for evaluating accuracies of dynamic link prediction methods and demonstrate why they can be misleading in many cases. We provide several recommendations on evaluating dynamic link prediction accuracy, including separation into two categories of evaluation. Finally we propose a unified metric to characterize link prediction accuracy effectively using a single number.