Improving Zero-Shot Entity Retrieval through Effective Dense Representations

TL;DR

This paper enhances zero-shot entity retrieval by improving candidate generation through a dense embedding approach, significantly increasing the likelihood of including the correct entity early in the candidate list.

Contribution

It introduces a simple BERT-based bi-encoder with a new pooling function and entity type info, achieving state-of-the-art accuracy in candidate generation for entity linking.

Findings

Achieved 84.28% accuracy on top-50 candidates on Zeshel dataset.

Outperformed previous methods with 82.06% accuracy on top-64 candidates.

Effective on both seen and unseen entity datasets.

Abstract

Entity Linking (EL) seeks to align entity mentions in text to entries in a knowledge-base and is usually comprised of two phases: candidate generation and candidate ranking. While most methods focus on the latter, it is the candidate generation phase that sets an upper bound to both time and accuracy performance of the overall EL system. This work's contribution is a significant improvement in candidate generation which thus raises the performance threshold for EL, by generating candidates that include the gold entity in the least candidate set (top-K). We propose a simple approach that efficiently embeds mention-entity pairs in dense space through a BERT-based bi-encoder. Specifically, we extend (Wu et al., 2020) by introducing a new pooling function and incorporating entity type side-information. We achieve a new state-of-the-art 84.28% accuracy on top-50 candidates on the Zeshel dataset, compared to the previous 82.06% on the top-64 of (Wu et al., 2020). We report the results from extensive experimentation using our proposed model on both seen and unseen entity datasets. Our results suggest that our method could be a useful complement to existing EL approaches.