PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval

TL;DR

The paper introduces PAIR, a novel dense passage retrieval method that leverages both query-centric and passage-centric similarity relations, significantly improving retrieval performance over previous models.

Contribution

It proposes a new approach that incorporates passage-centric similarity relations into dense retrieval, with formal formulations, pseudo-labeling via knowledge distillation, and a two-stage training process.

Findings

Outperforms state-of-the-art models on MSMARCO and Natural Questions datasets.

Effectively captures comprehensive similarity relations for better retrieval.

Demonstrates significant improvements in retrieval accuracy.

Abstract

Recently, dense passage retrieval has become a mainstream approach to finding relevant information in various natural language processing tasks. A number of studies have been devoted to improving the widely adopted dual-encoder architecture. However, most of the previous studies only consider query-centric similarity relation when learning the dual-encoder retriever. In order to capture more comprehensive similarity relations, we propose a novel approach that leverages both query-centric and PAssage-centric sImilarity Relations (called PAIR) for dense passage retrieval. To implement our approach, we make three major technical contributions by introducing formal formulations of the two kinds of similarity relations, generating high-quality pseudo labeled data via knowledge distillation, and designing an effective two-stage training procedure that incorporates passage-centric similarity relation constraint. Extensive experiments show that our approach significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions datasets.