Dynamic Injection of Entity Knowledge into Dense Retrievers

TL;DR

This paper introduces KPR, a BERT-based dense retriever that dynamically injects external entity knowledge through a novel attention mechanism, significantly improving retrieval accuracy for queries involving less-frequent entities without retraining.

Contribution

The paper presents KPR, a new method that enhances dense retrievers with dynamic, updatable entity embeddings and a context-entity attention layer, enabling better handling of entity-rich queries.

Findings

KPR outperforms baseline models on three datasets.

KPR achieves state-of-the-art results on two datasets.

KPR improves retrieval for less-frequent entities.

Abstract

Dense retrievers often struggle with queries involving less-frequent entities due to their limited entity knowledge. We propose the Knowledgeable Passage Retriever (KPR), a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings. This design enables KPR to incorporate external entity knowledge without retraining. Experiments on three datasets demonstrate that KPR consistently improves retrieval accuracy, with particularly large gains on the EntityQuestions dataset. When built on the off-the-shelf bge-base retriever, KPR achieves state-of-the-art performance among similarly sized models on two datasets. Models and code are released at https://github.com/knowledgeable-embedding/knowledgeable-embedding.