RaDeR: Reasoning-aware Dense Retrieval Models

TL;DR

RaDeR introduces reasoning-aware dense retrieval models trained on data from mathematical problem solving, significantly improving performance on reasoning tasks and outperforming traditional methods like BM25.

Contribution

RaDeR is the first dense retriever to outperform BM25 on reasoning steps, leveraging reasoning trajectories and self-reflective relevance evaluation for training.

Findings

RaDeR outperforms strong baselines on BRIGHT and RAR-b benchmarks.

RaDeR surpasses BM25 on Chain-of-Thought reasoning queries.

RaDeR achieves high performance with only 2.5% of the training data of REASONIR.

Abstract

We propose RaDeR, a set of reasoning-based dense retrieval models trained with data derived from mathematical problem solving using large language models (LLMs). Our method leverages retrieval-augmented reasoning trajectories of an LLM and self-reflective relevance evaluation, enabling the creation of both diverse and hard-negative samples for reasoning-intensive relevance. RaDeR retrievers, trained for mathematical reasoning, effectively generalize to diverse reasoning tasks in the BRIGHT and RAR-b benchmarks, consistently outperforming strong baselines in overall performance. Notably, RaDeR achieves significantly higher performance than baselines on the Math and Coding splits. In addition, RaDeR presents the first dense retriever that outperforms BM25 when queries are Chain-of-Thought reasoning steps, underscoring the critical role of reasoning-based retrieval to augment reasoning language models. Furthermore, RaDeR achieves comparable or superior performance while using only 2.5% of the training data used by the concurrent work REASONIR, highlighting the quality of our synthesized training data.