DeepRank: A New Deep Architecture for Relevance Ranking in Information Retrieval

TL;DR

DeepRank introduces a novel deep learning architecture that mimics human relevance judgment steps, effectively capturing IR characteristics and outperforming existing models on benchmark and large-scale data.

Contribution

The paper proposes DeepRank, a new deep learning model that explicitly models relevance detection, local relevance measurement, and aggregation, improving ranking performance.

Findings

DeepRank outperforms existing deep IR models on benchmark datasets.

DeepRank effectively captures IR characteristics like proximity and term importance.

Experimental results show significant improvement over previous methods.

Abstract

This paper concerns a deep learning approach to relevance ranking in information retrieval (IR). Existing deep IR models such as DSSM and CDSSM directly apply neural networks to generate ranking scores, without explicit understandings of the relevance. According to the human judgement process, a relevance label is generated by the following three steps: 1) relevant locations are detected, 2) local relevances are determined, 3) local relevances are aggregated to output the relevance label. In this paper we propose a new deep learning architecture, namely DeepRank, to simulate the above human judgment process. Firstly, a detection strategy is designed to extract the relevant contexts. Then, a measure network is applied to determine the local relevances by utilizing a convolutional neural network (CNN) or two-dimensional gated recurrent units (2D-GRU). Finally, an aggregation network with sequential integration and term gating mechanism is used to produce a global relevance score. DeepRank well captures important IR characteristics, including exact/semantic matching signals, proximity heuristics, query term importance, and diverse relevance requirement. Experiments on both benchmark LETOR dataset and a large scale clickthrough data show that DeepRank can significantly outperform learning to ranking methods, and existing deep learning methods.