ExpertRank: A Multi-level Coarse-grained Expert-based Listwise Ranking Loss

TL;DR

ExpertRank introduces a multi-level coarse-grained expert-based listwise ranking loss that enhances neural retrieval models by better exploiting candidate relevance signals through local prominence detection and expert combination.

Contribution

It proposes a novel ExpertRank loss that applies coarse graining and mixture of experts techniques to improve listwise ranking performance in neural retrieval models.

Findings

ExpertRank outperforms existing listwise losses on various neural retrieval models.

It achieves more reliable and competitive ranking results across different model complexities.

The approach effectively captures local prominence signals to enhance ranking accuracy.

Abstract

The goal of information retrieval is to recommend a list of document candidates that are most relevant to a given query. Listwise learning trains neural retrieval models by comparing various candidates simultaneously on a large scale, offering much more competitive performance than pairwise and pointwise schemes. Existing listwise ranking losses treat the candidate document list as a whole unit without further inspection. Some candidates with moderate semantic prominence may be ignored by the noisy similarity signals or overshadowed by a few especially pronounced candidates. As a result, existing ranking losses fail to exploit the full potential of neural retrieval models. To address these concerns, we apply the classic pooling technique to conduct multi-level coarse graining and propose ExpertRank, a novel expert-based listwise ranking loss. The proposed scheme has three major advantages: (1) ExpertRank introduces the profound physics concept of coarse graining to information retrieval by selecting prominent candidates at various local levels based on model prediction and inter-document comparison. (2) ExpertRank applies the mixture of experts (MoE) technique to combine different experts effectively by extending the traditional ListNet. (3) Compared to other existing listwise learning approaches, ExpertRank produces much more reliable and competitive performance for various neural retrieval models with different complexities, from traditional models, such as KNRM, ConvKNRM, MatchPyramid, to sophisticated BERT/ALBERT-based retrieval models.