Improving Neural Retrieval with Attribution-Guided Query Rewriting

TL;DR

This paper introduces an attribution-guided query rewriting method that uses token-level explanations to improve neural retrieval effectiveness, especially for ambiguous queries, by guiding LLM-based query clarification.

Contribution

It presents a novel approach combining token attribution and LLMs for query rewriting to enhance neural retrieval performance, addressing brittleness in existing methods.

Findings

Consistent improvement in retrieval effectiveness on BEIR datasets.

Larger gains observed for implicit or ambiguous queries.

Effective use of token attributions to guide query clarification.

Abstract

Neural retrievers are effective but brittle: underspecified or ambiguous queries can misdirect ranking even when relevant documents exist. Existing approaches address this brittleness only partially: LLMs rewrite queries without retriever feedback, and explainability methods identify misleading tokens but are used for post-hoc analysis. We close this loop and propose an attribution-guided query rewriting method that uses token-level explanations to guide query rewriting. For each query, we compute gradient-based token attributions from the retriever and then use these scores as soft guidance in a structured prompt to an LLM that clarifies weak or misleading query components while preserving intent. Evaluated on BEIR collections, the resulting rewrites consistently improve retrieval effectiveness over strong baselines, with larger gains for implicit or ambiguous information needs.