A Deep Relevance Matching Model for Ad-hoc Retrieval

TL;DR

This paper introduces a novel deep relevance matching model (DRMM) tailored for ad-hoc retrieval, addressing the unique challenges of relevance matching rather than semantic matching, and demonstrates its superior performance on benchmark datasets.

Contribution

The paper presents a new deep relevance matching model that effectively captures relevance signals for ad-hoc retrieval, differing from typical NLP matching models.

Findings

DRMM significantly outperforms traditional retrieval models.

DRMM surpasses state-of-the-art deep matching models.

Experimental results validate the effectiveness of relevance-specific features.

Abstract

In recent years, deep neural networks have led to exciting breakthroughs in speech recognition, computer vision, and natural language processing (NLP) tasks. However, there have been few positive results of deep models on ad-hoc retrieval tasks. This is partially due to the fact that many important characteristics of the ad-hoc retrieval task have not been well addressed in deep models yet. Typically, the ad-hoc retrieval task is formalized as a matching problem between two pieces of text in existing work using deep models, and treated equivalent to many NLP tasks such as paraphrase identification, question answering and automatic conversation. However, we argue that the ad-hoc retrieval task is mainly about relevance matching while most NLP matching tasks concern semantic matching, and there are some fundamental differences between these two matching tasks. Successful relevance matching requires proper handling of the exact matching signals, query term importance, and diverse matching requirements. In this paper, we propose a novel deep relevance matching model (DRMM) for ad-hoc retrieval. Specifically, our model employs a joint deep architecture at the query term level for relevance matching. By using matching histogram mapping, a feed forward matching network, and a term gating network, we can effectively deal with the three relevance matching factors mentioned above. Experimental results on two representative benchmark collections show that our model can significantly outperform some well-known retrieval models as well as state-of-the-art deep matching models.