PARADE: Passage Representation Aggregation for Document Reranking

TL;DR

PARADE introduces an effective passage aggregation method for document reranking using transformer models, significantly improving retrieval performance especially on broad information needs, with various strategies to enhance efficiency.

Contribution

The paper proposes PARADE, a novel passage representation aggregation technique that outperforms existing methods in document reranking tasks using transformer models.

Findings

PARADE significantly improves reranking results on broad information need collections.

Less complex aggregation techniques may be better for pinpointed information needs.

Efficiency strategies can enhance transformer-based aggregation methods.

Abstract

Pretrained transformer models, such as BERT and T5, have shown to be highly effective at ad-hoc passage and document ranking. Due to inherent sequence length limits of these models, they need to be run over a document's passages, rather than processing the entire document sequence at once. Although several approaches for aggregating passage-level signals have been proposed, there has yet to be an extensive comparison of these techniques. In this work, we explore strategies for aggregating relevance signals from a document's passages into a final ranking score. We find that passage representation aggregation techniques can significantly improve over techniques proposed in prior work, such as taking the maximum passage score. We call this new approach PARADE. In particular, PARADE can significantly improve results on collections with broad information needs where relevance signals can be spread throughout the document (such as TREC Robust04 and GOV2). Meanwhile, less complex aggregation techniques may work better on collections with an information need that can often be pinpointed to a single passage (such as TREC DL and TREC Genomics). We also conduct efficiency analyses, and highlight several strategies for improving transformer-based aggregation.