TongSearch-QR: Reinforced Query Reasoning for Retrieval

TL;DR

This paper introduces TongSearch QR, a family of small-scale language models that use reinforcement learning for reasoning-based query rewriting, achieving performance comparable to large models while being more practical for deployment.

Contribution

The work develops a semi-rule-based reward function and employs reinforcement learning to enable small models to perform reasoning-intensive query rewriting effectively.

Findings

TongSearch QR models outperform existing baselines on BRIGHT benchmark.

Small models achieve reasoning performance comparable to large-scale models.

The approach offers a practical solution for reasoning-intensive retrieval tasks.

Abstract

Traditional information retrieval (IR) methods excel at textual and semantic matching but struggle in reasoning-intensive retrieval tasks that require multi-hop inference or complex semantic understanding between queries and documents. One promising solution is to explicitly rewrite or augment queries using large language models (LLMs) to elicit reasoning-relevant content prior to retrieval. However, the widespread use of large-scale language models like GPT-4 or LLaMA3-70B remains impractical due to their high inference cost and limited deployability in real-world systems. In this work, we introduce TongSearch QR (Previously Known as "TongSearch Reasoner"), a family of small-scale language models for query reasoning and rewriting in reasoning-intensive retrieval. With a novel semi-rule-based reward function, we employ reinforcement learning approaches enabling smaller language models, e,g, Qwen2.5-7B-Instruct and Qwen2.5-1.5B-Instruct, to achieve query reasoning performance rivaling large-scale language models without their prohibitive inference costs. Experiment results on BRIGHT benchmark show that with BM25 as retrievers, both TongSearch QR-7B and TongSearch QR-1.5B models significantly outperform existing baselines, including prompt-based query reasoners and some latest dense retrievers trained for reasoning-intensive retrieval tasks, offering superior adaptability for real-world deployment.