Knowledge Base Completion Meets Transfer Learning

TL;DR

This paper introduces a transfer learning approach for knowledge base completion that leverages large-scale pre-training on unstructured text to improve fact prediction in structured knowledge bases, especially small datasets.

Contribution

It presents the first method for transferring knowledge between knowledge bases without entity or relation matching, applicable to both canonicalized and uncanonicalized data.

Findings

6% absolute increase in mean reciprocal rank on ReVerb20K

65% relative decrease in mean rank on ReVerb20K

Most impactful on small datasets without relying on large pre-trained models

Abstract

The aim of knowledge base completion is to predict unseen facts from existing facts in knowledge bases. In this work, we introduce the first approach for transfer of knowledge from one collection of facts to another without the need for entity or relation matching. The method works for both canonicalized knowledge bases and uncanonicalized or open knowledge bases, i.e., knowledge bases where more than one copy of a real-world entity or relation may exist. Such knowledge bases are a natural output of automated information extraction tools that extract structured data from unstructured text. Our main contribution is a method that can make use of a large-scale pre-training on facts, collected from unstructured text, to improve predictions on structured data from a specific domain. The introduced method is the most impactful on small datasets such as ReVerb20K, where we obtained 6% absolute increase of mean reciprocal rank and 65% relative decrease of mean rank over the previously best method, despite not relying on large pre-trained models like BERT.