Relational Graph Convolutional Networks Do Not Learn Sound Rules

TL;DR

This paper investigates whether R-GCN graph neural networks truly learn sound logical rules for knowledge graph completion, revealing they do not, which raises concerns about their explainability and generalization.

Contribution

The authors develop methods to extract and verify sound rules from R-GCN models and demonstrate these models do not learn sound rules despite high accuracy.

Findings

R-GCN models often do not learn sound rules.

No Datalog rule is sound for trained R-GCNs on benchmarks.

Proposed training variations can encourage learning sound rules.

Abstract

Graph neural networks (GNNs) are frequently used to predict missing facts in knowledge graphs (KGs). Motivated by the lack of explainability for the outputs of these models, recent work has aimed to explain their predictions using Datalog, a widely used logic-based formalism. However, such work has been restricted to certain subclasses of GNNs. In this paper, we consider one of the most popular GNN architectures for KGs, R-GCN, and we provide two methods to extract rules that explain its predictions and are sound, in the sense that each fact derived by the rules is also predicted by the GNN, for any input dataset. Furthermore, we provide a method that can verify that certain classes of Datalog rules are not sound for the R-GCN. In our experiments, we train R-GCNs on KG completion benchmarks, and we are able to verify that no Datalog rule is sound for these models, even though the models often obtain high to near-perfect accuracy. This raises some concerns about the ability of R-GCN models to generalise and about the explainability of their predictions. We further provide two variations to the training paradigm of R-GCN that encourage it to learn sound rules and find a trade-off between model accuracy and the number of learned sound rules.