Reevaluation of Inductive Link Prediction

TL;DR

This paper critically reevaluates inductive link prediction methods, revealing flaws in current evaluation protocols and proposing improved sampling techniques that lead to significantly different results.

Contribution

It identifies flaws in existing evaluation protocols for inductive link prediction and introduces an improved sampling method for more accurate assessment.

Findings

Current evaluation protocols are heavily flawed.

Simple baseline methods can outperform complex models under flawed protocols.

Reevaluation with improved protocols yields drastically different results.

Abstract

Within this paper, we show that the evaluation protocol currently used for inductive link prediction is heavily flawed as it relies on ranking the true entity in a small set of randomly sampled negative entities. Due to the limited size of the set of negatives, a simple rule-based baseline can achieve state-of-the-art results, which simply ranks entities higher based on the validity of their type. As a consequence of these insights, we reevaluate current approaches for inductive link prediction on several benchmarks using the link prediction protocol usually applied to the transductive setting. As some inductive methods suffer from scalability issues when evaluated in this setting, we propose and apply additionally an improved sampling protocol, which does not suffer from the problem mentioned above. The results of our evaluation differ drastically from the results reported in so far.