EviRerank: Adaptive Evidence Construction for Long-Document LLM Reranking

TL;DR

EviRerank introduces an adaptive evidence construction framework that improves long-document reranking efficiency and effectiveness for decoder-only LLMs by selectively summarizing relevant document parts.

Contribution

The paper presents EviRerank, a novel evidence-based reranking method that dynamically constructs compact contexts, outperforming existing full-document approaches in long-document LLM reranking tasks.

Findings

Outperforms full-document reranking baselines on multiple datasets.

Achieves state-of-the-art results on TREC DL'19 with significant improvements.

Reduces input length while maintaining high relevance scoring accuracy.

Abstract

Decoder-only LLM rerankers struggle with long documents: inference is costly and relevance signals can be diluted by irrelevant context. Motivated by an attention analysis indicating a consistent degradation trend when non-relevant text is appended, we propose EviRerank, an evidence-based long-document reranking framework for decoder-only LLMs. EviRerank (i) scores document blocks with a lightweight selector (BM25, bi-encoder, or cross-encoder), (ii) constructs a compact reranking context under a hard token cap by dynamically budgeting evidence blocks with Adaptive Evidence Budgeting (AEB) and adding a global summary cue via Summary Augmentation (SA), and (iii) reranks with a decoder-only LLM. Across TREC DL'19, DL'23, and MLDR-zh, EviRerank consistently outperforms full-document LLM reranking and strong block-selection baselines while substantially reducing the required input length. On TREC DL'19, EviRerank achieves 0.743 nDCG@10 and 0.307 MAP, establishing a new best result and improving over RankLLaMA (0.701/0.288) by +0.042 nDCG@10 (+6.0%) and +0.019 MAP (+6.6%).