CoRT: Complementary Rankings from Transformers
Marco Wrzalik, Dirk Krechel

TL;DR
CoRT is a neural first-stage ranking model that enhances traditional term-based methods like BM25 by leveraging pretrained language models, significantly improving candidate recall and enabling efficient passage retrieval.
Contribution
This paper introduces CoRT, a neural ranking model that complements BM25 using contextual embeddings, improving recall without adding significant query-time delay.
Findings
CoRT increases candidate recall when combined with BM25.
Subsequent re-rankers perform better with fewer candidates.
Passage retrieval with CoRT achieves low latency.
Abstract
Many recent approaches towards neural information retrieval mitigate their computational costs by using a multi-stage ranking pipeline. In the first stage, a number of potentially relevant candidates are retrieved using an efficient retrieval model such as BM25. Although BM25 has proven decent performance as a first-stage ranker, it tends to miss relevant passages. In this context we propose CoRT, a simple neural first-stage ranking model that leverages contextual representations from pretrained language models such as BERT to complement term-based ranking functions while causing no significant delay at query time. Using the MS MARCO dataset, we show that CoRT significantly increases the candidate recall by complementing BM25 with missing candidates. Consequently, we find subsequent re-rankers achieve superior results with less candidates. We further demonstrate that passage retrieval…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsAttention Is All You Need · Linear Layer · WordPiece · Softmax · Layer Normalization · Dropout · Attention Dropout · Residual Connection · Refunds@Expedia|||How do I get a full refund from Expedia? · Weight Decay
