Compositional Vector Space Models for Knowledge Base Completion

TL;DR

This paper introduces a compositional neural network approach for knowledge base completion that effectively reasons over multi-hop relations, enabling generalization to unseen paths and relation types, and demonstrates significant performance improvements.

Contribution

It presents a recursive neural network model that composes multi-hop relation embeddings for improved KB inference and zero-shot relation prediction.

Findings

Improves over traditional classifiers by 11%.

Outperforms pre-trained embedding methods by 7%.

Enables zero-shot relation prediction.

Abstract

Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X,Y) from bornIn(X,Y). Most previous methods infer simple one-hop relational synonyms like this, or use as evidence a multi-hop relational path treated as an atomic feature, like bornIn(X,Z) -> containedIn(Z,Y). This paper presents an approach that reasons about conjunctions of multi-hop relations non-atomically, composing the implications of a path using a recursive neural network (RNN) that takes as inputs vector embeddings of the binary relation in the path. Not only does this allow us to generalize to paths unseen at training time, but also, with a single high-capacity RNN, to predict new relation types not seen when the compositional model was trained (zero-shot learning). We assemble a new dataset of over 52M relational triples, and show that our method improves over a traditional classifier by 11%, and a method leveraging pre-trained embeddings by 7%.