R1-Ranker: Teaching LLM Rankers to Reason

TL;DR

This paper introduces R1-Ranker, a reinforcement learning framework that enhances large language models' reasoning abilities for ranking tasks across various domains, achieving state-of-the-art results and demonstrating the importance of iterative reasoning.

Contribution

The paper presents R1-Ranker, a novel reasoning-incentive framework with two designs, improving LLM ranking performance through reinforcement learning and iterative reasoning across multiple datasets.

Findings

IRanker-3B achieves state-of-the-art performance.

Over 15.7% average relative improvement across datasets.

Zero-shot out-of-domain performance improves by over 9%.

Abstract

Large language models (LLMs) have recently shown strong reasoning abilities in domains like mathematics, coding, and scientific problem-solving, yet their potential for ranking tasks, where prime examples include retrieval, recommender systems, and LLM routing, remains underexplored. Ranking requires complex reasoning across heterogeneous candidates, but existing LLM-based rankers are often domain-specific, tied to fixed backbones, and lack iterative refinement, limiting their ability to fully exploit LLMs' reasoning potential. To address these challenges, we propose R1-Ranker, a reasoning-incentive framework built on reinforcement learning, with two complementary designs: DRanker, which generates full rankings in one shot, and IRanker, which decomposes ranking into an iterative elimination process with step-wise rewards to encourage deeper reasoning. We evaluate unified R1-Rankers on nine datasets spanning recommendation, routing, and passage ranking, showing that IRanker-3B consistently achieves state-of-the-art performance, surpasses larger 7B models on some tasks, and yields a 15.7% average relative improvement. Ablation and generalization experiments further confirm the critical role of reinforcement learning and iterative reasoning, with IRanker-3B improving zero-shot performance by over 9% on out-of-domain tasks and reasoning traces boosting other LLMs by up to 22.87%. These results demonstrate that unifying diverse ranking tasks with a single reasoning-driven foundation model is both effective and essential for advancing LLM reasoning in ranking scenarios.