Reproducing Adaptive Reranking for Reasoning-Intensive IR

TL;DR

This paper reproduces and evaluates GAR, a graph-based reranking method, in reasoning-intensive retrieval tasks, demonstrating its effectiveness with minimal added computational costs.

Contribution

It adapts and tests GAR for reasoning-intensive retrieval, showing it improves effectiveness across models without significant computational overhead.

Findings

GAR boosts retrieval effectiveness in reasoning tasks

The quality of reranker's signal is crucial for success

GAR contributes minimally to computational costs

Abstract

The classical cascading pipeline of retrieve--rerank suffers from a bounded recall problem, stemming from limitations of the first-stage retriever. Most current approaches address the bounded recall problem by improving the first-stage retriever, but this incurs substantial training and inference costs, especially to handle queries that require substantial reasoning. To circumvent the computational costs of reasoning-based retrievers, we replicate the findings of GAR, Graph-based Adaptive Reranking, on the BRIGHT reasoning-intensive retrieval benchmark. GAR addresses the bounded recall problem by modifying the reranking process itself through iterative exploration of a corpus graph, but it was previously only tested on models designed for topical and question-answering-style queries. Hence, reproduce GAR in reasoning-intensive settings with reasoning and non-reasoning reranking models. We observe that the quality of the reranker's signal plays an important role in identifying additional relevant documents within the corpus graph. Overall, we find that GAR boosts the effectiveness of reasoning-intensive retrieval across a variety of models while contributing minimally to computational overheads. Ultimately, this work enables more practical deployment of retrieval systems that can address reasoning-intensive queries.