Enhancing Documents with Multidimensional Relevance Statements in Cross-encoder Re-ranking

TL;DR

This paper introduces a method to improve document re-ranking by incorporating multidimensional relevance statements, specifically including credibility, to enhance retrieval effectiveness beyond traditional topicality-focused models.

Contribution

The paper proposes a novel approach that enhances candidate documents with relevance statements for multiple dimensions, improving cross-encoder re-ranking performance.

Findings

Outperforms aggregation-based re-rankers

Statistically significant improvements in effectiveness

Effective in consumer health search context

Abstract

In this paper, we propose a novel approach to consider multiple dimensions of relevance beyond topicality in cross-encoder re-ranking. On the one hand, current multidimensional retrieval models often use na\"ive solutions at the re-ranking stage to aggregate multiple relevance scores into an overall one. On the other hand, cross-encoder re-rankers are effective in considering topicality but are not designed to straightforwardly account for other relevance dimensions. To overcome these issues, we envisage enhancing the candidate documents -- which are retrieved by a first-stage lexical retrieval model -- with "relevance statements" related to additional dimensions of relevance and then performing a re-ranking on them with cross-encoders. In particular, here we consider an additional relevance dimension beyond topicality, which is credibility. We test the effectiveness of our solution in the context of the Consumer Health Search task, considering publicly available datasets. Our results show that the proposed approach statistically outperforms both aggregation-based and cross-encoder re-rankers.