CEAR: Cross-Entity Aware Reranker for Knowledge Base Completion

TL;DR

This paper introduces CEAR, a novel BERT-based reranker that improves knowledge base completion by jointly scoring entities with cross-entity attention, surpassing previous methods on the OLPBench dataset.

Contribution

The paper presents CEAR, a cross-entity aware reranker that leverages BERT's factual knowledge to enhance the accuracy of knowledge base completion models.

Findings

CEAR achieves state-of-the-art results on the OLPBench dataset.

Joint scoring of entities with BERT improves KBC performance.

CEAR outperforms previous embedding-based and BERT-based methods.

Abstract

Pre-trained language models (LMs) like BERT have shown to store factual knowledge about the world. This knowledge can be used to augment the information present in Knowledge Bases, which tend to be incomplete. However, prior attempts at using BERT for task of Knowledge Base Completion (KBC) resulted in performance worse than embedding based techniques that rely only on the graph structure. In this work we develop a novel model, Cross-Entity Aware Reranker (CEAR), that uses BERT to re-rank the output of existing KBC models with cross-entity attention. Unlike prior work that scores each entity independently, CEAR uses BERT to score the entities together, which is effective for exploiting its factual knowledge. CEAR achieves a new state of art for the OLPBench dataset.