Interpreting Embedding Models of Knowledge Bases: A Pedagogical Approach

TL;DR

This paper introduces a pedagogical method to interpret embedding models of knowledge bases by extracting weighted Horn rules, addressing the challenge of understanding their predictions in large-scale relational data.

Contribution

It adapts pedagogical approaches from neural network interpretability to the context of knowledge base embeddings, enabling rule extraction for better understanding.

Findings

Pedagogical methods can effectively extract interpretable rules from embedding models.

The approach reveals strengths and weaknesses of embedding models in knowledge base completion.

Interpretability techniques need adaptation for large-scale relational data.

Abstract

Knowledge bases are employed in a variety of applications from natural language processing to semantic web search; alas, in practice their usefulness is hurt by their incompleteness. Embedding models attain state-of-the-art accuracy in knowledge base completion, but their predictions are notoriously hard to interpret. In this paper, we adapt "pedagogical approaches" (from the literature on neural networks) so as to interpret embedding models by extracting weighted Horn rules from them. We show how pedagogical approaches have to be adapted to take upon the large-scale relational aspects of knowledge bases and show experimentally their strengths and weaknesses.