A*Net and NBFNet Learn Negative Patterns on Knowledge Graphs

TL;DR

This paper compares rule-based and GNN models NBFNet and A*Net for knowledge graph completion, revealing that negative patterns hidden from rule-based methods significantly influence performance differences.

Contribution

It uncovers the role of hidden negative patterns in explaining performance gaps between rule-based and GNN models in knowledge graph completion.

Findings

Negative patterns explain performance differences

Models benefit from penalizing incorrect facts

GNNs outperform rule-based methods on benchmarks

Abstract

In this technical report, we investigate the predictive performance differences of a rule-based approach and the GNN architectures NBFNet and A*Net with respect to knowledge graph completion. For the two most common benchmarks, we find that a substantial fraction of the performance difference can be explained by one unique negative pattern on each dataset that is hidden from the rule-based approach. Our findings add a unique perspective on the performance difference of different model classes for knowledge graph completion: Models can achieve a predictive performance advantage by penalizing scores of incorrect facts opposed to providing high scores for correct facts.