Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking

TL;DR

This paper introduces a novel method for passage re-ranking that incorporates explicit knowledge from incomplete and noisy knowledge graphs using knowledge graph distillation, a knowledge meta graph, and dynamic knowledge injection, improving performance especially on domain-specific queries.

Contribution

It presents a new approach combining PLMs and knowledge graphs with a distillation method and knowledge injector for improved passage re-ranking.

Findings

Enhanced re-ranking accuracy on domain-specific queries.

Effective integration of noisy knowledge graphs.

Improved handling of vocabulary mismatch issues.

Abstract

Passage re-ranking is to obtain a permutation over the candidate passage set from retrieval stage. Re-rankers have been boomed by Pre-trained Language Models (PLMs) due to their overwhelming advantages in natural language understanding. However, existing PLM based re-rankers may easily suffer from vocabulary mismatch and lack of domain specific knowledge. To alleviate these problems, explicit knowledge contained in knowledge graph is carefully introduced in our work. Specifically, we employ the existing knowledge graph which is incomplete and noisy, and first apply it in passage re-ranking task. To leverage a reliable knowledge, we propose a novel knowledge graph distillation method and obtain a knowledge meta graph as the bridge between query and passage. To align both kinds of embedding in the latent space, we employ PLM as text encoder and graph neural network over knowledge meta graph as knowledge encoder. Besides, a novel knowledge injector is designed for the dynamic interaction between text and knowledge encoder. Experimental results demonstrate the effectiveness of our method especially in queries requiring in-depth domain knowledge.