ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking

TL;DR

ERank is a novel text reranking model that combines supervised fine-tuning and reinforcement learning to achieve high relevance discrimination and global ranking awareness efficiently.

Contribution

The paper introduces ERank, a two-stage training pipeline that enhances pointwise rerankers with generative scoring and listwise reinforcement learning.

Findings

ERank outperforms existing rerankers on multiple benchmarks.

ERank-4B achieves 38.7 nDCG@10 on BRIGHT.

ERank-32B sets a new state-of-the-art with 40.2 nDCG@10.

Abstract

Text reranking models are a crucial component in modern systems like Retrieval-Augmented Generation, tasked with selecting the most relevant documents prior to generation. However, current Large Language Models (LLMs) powered rerankers often face a fundamental trade-off. On one hand, Supervised Fine-Tuning based pointwise methods that frame relevance as a binary classification task lack the necessary scoring discrimination, particularly for those built on reasoning LLMs. On the other hand, approaches designed for complex reasoning often employ powerful yet inefficient listwise formulations, rendering them impractical for low latency applications. To resolve this dilemma, we introduce ERank, a highly effective and efficient pointwise reranker built from a reasoning LLM that excels across diverse relevance scenarios. We propose a novel two-stage training pipeline that begins with Supervised Fine-Tuning (SFT). In this stage, we move beyond binary labels and train the model generatively to output fine grained integer scores, which significantly enhances relevance discrimination. The model is then further refined using Reinforcement Learning (RL) with a novel, listwise derived reward. This technique instills global ranking awareness into the efficient pointwise architecture. We evaluate the ERank reranker on the BRIGHT, FollowIR, TREC DL, and BEIR benchmarks, demonstrating superior effectiveness and robustness compared to existing approaches. On the reasoning-intensive BRIGHT benchmark, our ERank-4B achieves an nDCG@10 of 38.7, while a larger 32B variant reaches a state of the art nDCG@10 of 40.2.

Tables

Table 1. Table 1: Average nDCG@10 of QwQ-32B reasoning LLM when varying scoring discrimination on three benchmarks.

Scoring	BRIGHT	TREC DL	BEIR (5 subsets)
yes / no	20.8	60.9	31.1
$\{0,1,\cdots ,3\}$	22.7	64.1	36.5
$\{0,1,\cdots ,10\}$	23.2	66.1	37.1

Table 2. Table 2: Statistics of training data.

Stage	# Queries	# Doc per query	# Query-doc pairs
SFT	14,799	20	295,980
RL	2,048	20	40,960

Table 3. Table 3: Evaluation on different relevance types using original queries without hybrid scores. Best results are indicated in bold, while second-best results are underlined.

Method		Average	Reasoning-intensive Relevance		Semantic Relevance
			BRIGHT	FollowIR	BEIR	TREC DL
First-stage retriever		25.9	13.7	0	40.8	49.3
Listwise	Rank-R1-7B (Zhuang et al. 2025)	34.6	15.7	3.6	49.0	70.0
	Rearank-7B (Zhang et al. 2025)	35.3	17.4	2.3	49.0	72.5
Pointwise	JudgeRank-8B (Niu et al. 2024)	32.1	17.0	9.9	39.1	62.6
	Rank1-7B (Weller et al. 2025b)	34.6	18.2	9.1	44.2	67.1
	QwQ-32B powered reranker (Ours)	37.7	23.2	14.1	47.5	66.1
	ERank-4B (Ours)	36.8	22.7	11.0	44.8	68.9
	ERank-14B (Ours)	36.9	23.1	10.3	47.1	67.1
	ERank-32B (Ours)	38.1	24.4	12.1	47.7	68.1

Table 4. Table 4: Further evaluation on BRIGHT benchmark. *Taken from BRIGHT online website (Su et al. 2025 ) .

Method	nDCG@10
Retrieve by BM25 using GPT-4 reason-query
BM25	27.0
Rank-R1-7B (Zhuang et al. 2025)	23.9
Rank1-7B (Weller et al. 2025b)	25.5
Rearank-7B (Zhang et al. 2025)	29.1
XRR2-Gemini-2.5-Flash (jataware 2025)	40.3*
JudgeRank-8B (Niu et al. 2024)	24.4
+ BM25 hybrid	31.0
ERank-4B (Ours)	32.9
+ BM25 hybrid	36.1
ERank-14B (Ours)	33.5
+ BM25 hybrid	36.7
ERank-32B (Ours)	34.6
+ BM25 hybrid	37.4
Retrieve by ReasonIR-8B using GPT-4 reason-query
ReasonIR-8B (Shao et al. 2025)	30.5
Rank-R1-7B (Zhuang et al. 2025)	24.1
Rank1-7B (Weller et al. 2025b)	24.3
Rearank-7B (Zhang et al. 2025)	27.5
JudgeRank-8B (Niu et al. 2024)	20.2
+ BM25 hybrid	22.7
Rank-R1-32B-v0.2 (ielabgroup 2025)	37.7*
+ BM25 hybrid	40.0*
ERank-4B (Ours)	30.5
+ BM25 hybrid	38.7
ERank-14B (Ours)	31.8
+ BM25 hybrid	39.3
ERank-32B (Ours)	32.8
+ BM25 hybrid	40.2

Table 5. Table 5: Performance of different training stages.

	Avg.	BRIGHT	FollowIR	BEIR-5	TREC DL
Qwen3-4B	12.7	3.6	1.9	6.4	39.0
SFT Only	32.8	22.0	11.2	30.0	68.1
RL Only	31.8	20.1	12.2	28.2	66.5
SFT+RL	33.8	22.7	11.0	32.4	68.9

Table 6. Table 6: Performance of different rewards for RL training.

	Avg.	BRIGHT	FollowIR	BEIR-5	TREC DL
Before RL	32.8	22.0	11.2	30.0	68.1
$r_{\text{SE}}$	31.1	22.3	8.7	30.8	62.6
$r_{\text{nDCG}}$	33.8	21.5	13.2	32.8	67.6
$r_{\text{RR}}$	33.8	22.7	11.0	32.4	68.9

Table 7. Table 7: Task-specific instruction used in prompts.

Benchmark (Subset)	Instruction
BRIGHT (AoPS)	We want to find different but similar math problems to the query. A document is relevant if it uses the same class of functions and shares any overlapping techniques.
BRIGHT (LeetCode)	I am looking to find different problems that share similar data structures (of any kind) or algorithms (e.g. DFS, DP, sorting, traversals, etc.). I am looking for problems that share one or both of these similarities to the query. Does the passage below share any similarities? e.g. if there was a textbook on leetcode problems, this would be in the same book even though it could be in a different chapter.
BRIGHT (Pony)	I will use the programming language pony. But to solve the problem above, I need to know things about pony. A passage is relevant if it contains docs that match any part (even basic parts) of the code I will have to write for the above program.
BRIGHT (TheoremQA-Q)	We want to find a document which uses the same mathematical process as the query. A document is relevant if it uses the same mathematical process as the query.
BRIGHT (TheoremQA-T)	We want to find a document which uses the same mathematical process as the query. A document is relevant if it uses the same mathematical process as the query.
BRIGHT (others)	A document is relevant if it contains information that helps answer or address the query. A document is not relevant if it doesn’t contain information that helps answer the query, even if it mentions similar topics.
BEIR / TREC DL	Given a query, retrieval relevant passage.
FollowIR	Retrieval the relevant passage for the given query. Be careful about the extra requirements about relevance in the query.

Table 8. Table 8: SFT configurations used in LLaMA-Factory, while those not mentioned are kept as the default values.

	Configuration	Value
Shared	finetuning_type	lora
	lora_rank	32
	lora_alpha	64
	lora_target	all
	cutoff_len	2048
	learning_rate	1e-4
	num_train_epochs	1.0
	lr_scheduler_type	cosine
	warmup_ratio	0.05
	bf16	true
For Qwen3-4B-Base	template	default
	per_device_train_batch_size	8
	gradient_accumulation_steps	2
	num_gpus	8
For Qwen3-14B-Base	template	default
	per_device_train_batch_size	1
	gradient_accumulation_steps	8
	num_gpus	16
For Qwen3-32B	template	qwen3
	per_device_train_batch_size	1
	gradient_accumulation_steps	8
	num_gpus	16

Table 9. Table 9: RL configurations used in verl, and those not mentioned are kept as the default values.

	Configuration	Value
Shared	train batch size	1280
	max_prompt_length	1024
	max_response_length	1024
	learning_rate	1e-6
	mini_batch_size	320
	clip_ratio ($ϵ$)	0.2
	use_kl_loss	True
	kl_loss_coef ($\beta$)	0.001
	kl_loss_type	low_var_kl
	rollout_n ($G$)	5
	gpus_per_node	8
	total_epochs	10
For 4B LLM	nnodes	1
	micro_batch_size_per_gpu	20
For 14B LLM	nnodes	2
	micro_batch_size_per_gpu	10
For 32B LLM	nnodes	2
	micro_batch_size_per_gpu	10

Table 10. Table 10: Detailed p p -MRR on each subset of FollowIR benchmark.

Method	Average	Core17	News21	Robust04
BM25	0.0	0.0	0.0	0.0
JudgeRank-8B	9.9	14.6	2.0	13.1
Rank-R1-7B	3.6	6.1	4.0	0.7
Rank1-7B	9.1	10.1	4.8	12.3
Rearank-7B	2.3	4.3	0.1	2.5
QwQ-32B	14.1	20.9	9.4	12.0
ERank-4B	11.0	17.5	1.3	14.2
ERank-14B	10.3	13.6	2.0	15.4
ERank-32B	12.1	19.0	1.7	15.5

Table 11. Table 11: Detailed nDCG@10 on each subset of TREC DL benchmark.

Method	Average	TREC DL19	TREC DL20
BM25	49.3	50.6	48.0
JudgeRank-8B	62.6	65.4	59.9
Rank-R1-7B	70.0	72.2	67.7
Rank1-7B	67.1	69.0	65.1
Rearank-7B	72.5	74.8	70.1
QwQ-32B	66.1	67.5	64.8
ERank-4B	68.9	71.1	66.7
ERank-14B	67.1	68.9	65.4
ERank-32B	68.1	70.8	65.5

Table 12. Table 12: Detailed nDCG@10 on each subset of BRIGHT benchmark. *Taken from BRIGHT online website.

Method	Avg.	StackExchange							Coding		Theorem-based
		Bio.	Earth.	Econ.	Psy.	Rob.	Stack.	Sus.	Leet.	Pony	AoPS	TheoQ.	TheoT.
Retrieve top 100 documents by BM25, using original query
BM25	13.7	18.2	27.9	16.5	13.4	10.9	16.3	16.1	24.7	4.3	6.5	7.3	2.1
Rank-R1-7B	15.7	23.4	29.2	16.4	23.0	17.0	10.9	25.9	15.8	4.8	5.8	7.1	9.3
Rank1-7B	18.2	31.6	34.4	18.0	23.5	16.7	18.6	22.9	20.1	9.4	4.5	9.4	9.9
Rearank-7B	17.4	23.2	26.7	17.2	22.7	18.2	16.7	25.3	26.8	7.2	7.5	7.7	9.7
JudgeRank-8B	17.0	28.7	32.2	20.9	24.6	16.5	18.3	20.6	11.7	7.1	4.7	8.4	10.0
+ BM25 hybrid	19.0	28.3	36.5	21.9	24.1	15.3	22.7	23.5	25.1	6.8	6.7	8.3	8.5
QwQ-32B	23.2	32.7	44.7	23.9	30.5	21.6	23.8	23.8	25.7	17.3	12.7	11.2	11.1
+ BM25 hybrid	23.9	33.0	46.5	25.3	28.2	21.1	25.6	25.3	28.7	17.2	13.0	11.8	10.7
ERank-4B	22.7	30.4	42.5	21.5	27.7	22.4	22.9	24.0	31.6	14.6	11.0	12.1	11.4
+ BM25 hybrid	23.9	32.7	45.4	23.1	29.2	21.8	24.7	25.6	33.4	15.6	12.2	12.4	10.5
ERank-14B	23.1	31.2	43.6	25.8	27.8	23.1	23.9	24.6	29.8	16.8	8.6	10.5	11.9
+ BM25 hybrid	24.6	32.7	45.8	27.2	29.4	24.1	25.6	26.5	32.7	17.5	10.5	12.1	11.9
ERank-32B	24.4	33.5	44.5	23.9	29.4	23.8	27.1	26.4	32.5	15.8	12.5	10.9	12.2
+ BM25 hybrid	25.4	35.1	46.2	25.5	29.4	24.2	27.5	27.6	34.9	16.7	13.2	11.5	12.5
Retrieve top 100 documents by BM25, using GPT4 reason-query
BM25	27.0	53.6	54.1	24.3	38.7	18.9	27.7	26.3	19.3	17.6	3.9	19.2	20.8
Rank-R1-7B	23.9	38.2	29.4	23.4	33.0	24.9	14.9	33.2	18.2	16.1	3.8	16.6	34.8
Rank1-7B	25.5	45.8	37.0	22.2	31.7	20.6	23.0	34.2	15.7	19.8	1.3	19.8	34.7
Rearank-7B	29.1	42.0	37.5	26.4	39.1	25.0	25.1	32.6	26.2	29.2	5.9	28.0	32.2
XRR-Gemini-2.5-Flash*	40.3	63.1	55.4	38.5	52.9	37.1	38.2	44.6	21.9	35.0	15.7	34.4	46.2
JudgeRank-8B	24.4	41.4	34.7	26.2	36.0	24.0	27.6	26.1	10.2	14.2	3.4	20.3	28.9
+ BM25 hybrid	31.0	55.3	53.4	31.4	41.6	26.7	32.8	33.3	19.6	19.5	3.7	23.4	30.9
ERank-4B	32.9	48.2	46.7	30.0	43.1	28.4	31.5	38.1	28.5	23.5	10.4	26.9	39.0
+ BM25 hybrid	36.1	58.5	55.6	32.6	47.2	30.0	34.7	40.6	28.9	25.8	11.2	28.9	39.0
ERank-14B	33.5	51.4	48.6	30.8	41.3	26.7	35.6	39.1	27.3	26.4	10.9	25.7	37.9
+ BM25 hybrid	36.7	59.9	57.3	34.8	46.7	29.5	36.9	41.2	29.4	28.7	10.5	28.0	38.1
ERank-32B	34.6	55.5	49.1	30.4	44.7	27.9	35.6	40.6	29.2	24.2	10.4	27.6	40.0
+ BM25 hybrid	37.4	62.9	57.5	33.2	48.4	30.5	36.5	42.3	32.7	25.4	10.8	28.7	40.4
Retrieve top 100 documents by ReasonIR-8B, using GPT4 reason-query
ReasonIR-8B	30.5	43.5	43.0	32.8	38.9	21.1	30.6	27.3	31.6	19.6	7.3	34.1	36.7
Rank-R1-7B	24.1	39.3	28.1	23.9	30.0	17.3	18.1	33.2	18.6	15.0	4.2	25.4	35.7
Rank1-7B	24.3	44.1	33.5	21.8	30.0	15.0	22.1	28.5	11.8	21.7	1.2	26.2	36.2
Rearank-7B	27.5	35.3	29.8	25.5	35.7	19.1	20.1	32.9	29.9	20.2	6.2	36.7	38.3
JudgeRank-8B	20.2	37.1	27.2	19.2	28.6	11.6	19.9	22.5	10.2	10.2	3.6	22.9	29.4
+ BM25 hybrid	22.7	40.4	28.9	22.3	35.5	14.2	23.0	25.7	11.8	10.6	3.6	25.2	31.1
Rank-R1-v0.2-32B*	37.7	60.1	56.3	36.6	52.1	30.2	37.6	45.9	25.5	14.6	10.1	38.6	44.3
+BM25 hybrid*	40.0	64.4	60.1	38.3	52.2	30.7	40.6	46.7	33.3	17.4	10.1	38.6	47.7
ERank-4B	30.5	42.1	42.5	26.3	36.4	20.8	27.3	33.2	31.7	21.8	10.9	32.8	40.6
+ BM25 hybrid	38.7	58.7	56.6	33.8	48.7	29.1	38.2	40.8	32.7	32.0	9.8	35.2	48.4
ERank-14B	31.8	46.6	42.5	25.2	37.3	19.6	30.2	34.6	31.9	25.6	10.5	32.4	45.0
+ BM25 hybrid	39.3	60.1	55.8	34.2	49.5	28.4	40.1	41.2	33.3	34.0	11.0	35.7	48.5
ERank-32B	32.8	49.3	43.4	28.4	36.8	20.8	32.8	34.6	36.0	22.3	11.3	34.4	43.5
+ BM25 hybrid	40.2	61.5	56.6	36.5	49.4	28.9	41.8	42.7	36.0	31.6	11.4	37.0	49.1

Table 13. Table 13: Detailed nDCG@10 on each subset of BEIR benchmark.

Method	Avg.	ArguA.	CliF.	CQA.	DBP.	Fever	FiQA	HotP.	MSM.	NFC.	NQ	Quora	SciD.	SciF.	TREC-C.	TouC.
BM25	40.8	30.0	16.5	30.2	31.8	65.1	23.6	63.3	22.8	32.2	30.6	78.9	14.9	67.9	59.5	44.2
JudgeRank-8B	39.1	13.4	14.8	32.0	34.5	47.4	30.2	53.3	27.2	34.1	46.5	65.8	15.2	64.5	81.6	25.6
Rank-R1-7B	49.0	26.0	24.8	39.8	42.7	78.6	38.9	67.8	35.1	36.2	55.8	84.2	18.8	74.9	82.2	29.0
Rank1-7B	44.2	20.6	16.7	33.5	37.5	66.4	37.8	62.7	31.0	35.6	53.1	70.2	17.0	76.8	79.8	25.1
Rearank-7B	49.0	32.1	21.9	41.0	45.2	73.8	36.6	72.3	35.5	32.7	54.5	79.1	20.2	74.8	79.7	36.1
QwQ-32B	47.5	51.7	16.2	37.9	40.7	68.0	37.1	59.0	30.5	36.3	50.9	78.0	19.7	75.7	82.4	28.9
ERank-4B	44.8	36.7	17.8	38.6	39.1	56.8	34.3	53.7	34.4	35.1	51.8	74.0	17.1	73.1	79.6	29.9
ERank-14B	47.1	47.6	18.5	38.5	38.5	65.2	36.2	56.9	34.7	35.0	52.1	78.7	17.4	74.2	81.8	31.3
ERank-32B	47.7	49.1	18.7	38.4	38.8	67.1	37.0	57.6	34.8	36.7	52.4	78.6	17.6	73.9	82.4	32.1

Equations

$$\pi (c_i,s_i\mid I,q,d_i),i=1,2,\dots ,N.$$

$$\frac{\text{Pr(token=yes)}}{\text{Pr(token=yes) + Pr(token=no)}}.$$

$${\mathcal{L}}_{SFT}(\theta )=-\sum _{(x,y)\in \mathcal{D}}\log P(y\mid x;\theta ).$$

$$J_{GRPO}(\theta )={\mathbb{E}}_{x\sim \mathcal{D},\{y_i\}\sim {\pi }_{{\theta }_{\text{old}}}}\left[\frac{1}{G}\sum _{i=1}^G\frac{1}{|y_i|}\sum _{t=1}^{|y_i|}\left(\mathcal{C}-\beta D_{\text{KL}}\right)\right],$$

$$\mathcal{C}=\min$$

$$\text{clip}\left(\frac{{\pi }_{\theta }(y_{i,t}|x,y_{i,<t})}{{\pi }_{{\theta }_{\text{old}}}(y_{i,t}|x,y_{i,<t})},1-ϵ,1+ϵ\right){\hat{A}}_{i,t}),$$

$$D_{\text{KL}}=\frac{{\pi }_{\text{ref}}(y_{i,t}|x,y_{i,<t})}{{\pi }_{\theta }(y_{i,t}|x,y_{i,<t})}-\log \frac{{\pi }_{\text{ref}}(y_{i,t}|x,y_{i,<t})}{{\pi }_{\theta }(y_{i,t}|x,y_{i,<t})}-1,$$

$$r_{\text{RR}}=\{\begin{array}{ll}\frac{1}{{\varphi }_i^{(j)}},& \text{formatted, }{d}_i\in {\mathcal{D}}^P,\\ -\frac{1}{{\Phi }_{{\mathcal{D}}^P}^{\text{(min)}}},& \text{formatted, }{d}_i\in {\mathcal{D}}^N,{\varphi }_i^{(j)}\le {\Phi }_{{\mathcal{D}}^P}^{\text{(max)}},\\ 1-\frac{(s_i-t_i{)}^2}{100},& \text{formatted, }{d}_i\in {\mathcal{D}}^N,{\varphi }_i^{(j)}>{\Phi }_{{\mathcal{D}}^P}^{\text{(max)}},\\ -1,& \text{otherwise},\end{array}$$

$$r_{\text{SE}}=\{\begin{array}{ll}1-\frac{(s_i-t_i{)}^2}{100},& \text{formatted},\\ -1,& \text{otherwise},\end{array}$$

$$\text{DCG}=\sum _{i=1}^N\sum _{j=1}^G\frac{2^{re{l}_i}-1}{{\log }_2\left({\varphi }_i^{(j)}+1\right)},$$

$$\text{DCG}=\sum _{i=1}^N\sum _{j=1}^G\mathbb{I}(d_i\in {\mathcal{D}}^P)f({\varphi }_i^{(j)}).$$

$$\text{nDCG}=\frac{\text{DCG}}{\text{IDCG}}.$$

$$r_{\text{nDCG}}=\{\begin{array}{ll}\frac{f({\varphi }_i^{(j)})}{IDCG},& \text{formatted, }{d}_i\in {\mathcal{D}}^P,\\ -\frac{f({\Phi }_{{\mathcal{D}}^P}^{\text{(min)}})}{IDCG},& \text{formatted, }{d}_i\in {\mathcal{D}}^N,{\varphi }_i^{(j)}\le {\Phi }_{{\mathcal{D}}^P}^{\text{(max)}},\\ 1-\frac{(s_i-t_i{)}^2}{100},& \text{formatted, }{d}_i\in {\mathcal{D}}^N,{\varphi }_i^{(j)}>{\Phi }_{{\mathcal{D}}^P}^{\text{(max)}},\\ -1,& \text{otherwise},\end{array}$$

Full text

1 Introduction

Text reranking is a fundamental component of various Natural Language Processing and Information Retrieval applications, utilized extensively in downstream tasks such as open-domain question answering (Lee et al. 2018), web search (Lin, Nogueira, and Yates 2022), and recommendation systems (Chuang et al. 2020; Gao et al. 2025). Large Language Models (LLMs) have significantly reshaped the text reranking landscape. On one hand, studies have sought to leverage the advanced text understanding capabilities of LLMs for reranking, either through zero-shot prompting or Supervised Fine-Tuning (Zhang et al. 2023; Liu et al. 2024). On the other hand, LLMs have introduced new application paradigms like Retrieval-Augmented Generation (Wu et al. 2024; Gupta, Ranjan, and Singh 2024; Wang et al. 2024) and agentic systems (Huang et al. 2024; Li et al. 2024). These paradigms demand capabilities beyond traditional semantic relevance, requiring models to perform reasoning-intensive retrieval, such as identifying issue-relevant code snippets to resolve a specific programming problem. Recent advancements in test-time compute (OpenAI 2024; DeepSeek AI 2025) have shown promise in such scenarios, with a growing number of text rerankers based on reasoning LLMs (Weller et al. 2025b; Zhuang et al. 2025; Zhang et al. 2025).

Prior approaches have generally treated traditional semantic relevance and reasoning-intensive reranking as distinct challenges, which are illustrated in Figure 1. For semantic tasks, Supervised Fine-Tuning (SFT) is a common strategy (Ma et al. 2024; Sun et al. 2023; Zhang et al. 2024). However, most SFT-based rerankers adopt a pointwise scoring method based on binary classification, where the model predicts labels like “Relevant” or “Not Relevant”. We argue this approach is suboptimal as it leads to poor score discrimination, a problem exacerbated in modern reasoning LLMs that generate overconfident predictions after Chain-of-Thought (CoT). For reasoning-intensive tasks, Reinforcement Learning (RL) has shown promise (Zhuang et al. 2025; Zhang et al. 2025). However, these methods often rely on listwise or setwise formulations and ingest multiple candidate documents simultaneously. With sliding windows, they process different batches of documents sequentially, resulting in prohibitive latency and memory footprints that make them impractical for real-world deployment.

This work addresses a central question: can a single, efficient reranker powered by reasoning LLM be trained to excel at both semantic relevance and deep reasoning? We contend that this is achievable by enhancing the pointwise architecture, which scores each document independently. We introduce a novel, two-stage training framework illustrated in Figure 4, which seamlessly integrates Supervised Fine-Tuning (SFT) with Reinforcement Learning (RL) for LLM-based reranker training. The first stage, SFT, trains a base model on a diverse mixture of semantic and reasoning-oriented data. Crucially, we abandon the standard binary classification paradigm and instead train the model using a fine-grained integer scoring scheme, which fully utilizes the generative power of LLMs and significantly improves score discrimination. We also employ a data synthesis strategy to generate high-quality reasoning chains and fine-grained scores to overcome data scarcity. In the second stage, we further refine the SFT-tuned model using RL. To bridge the gap between listwise optimality and pointwise efficiency, we introduce a novel, listwise-derived reward function. This function provides a global ranking signal during training, encouraging the model to learn the relative importance of documents. This allows our pointwise model to benefit from listwise-style optimization while retaining its low latency.

Extensive experiments on semantic (TREC DL (Craswell et al. 2020, 2021), BEIR (Thakur et al. 2021)) and reasoning-intensive benchmarks (BRIGHT (Su et al. 2024), FollowIR (Weller et al. 2025a)) confirm that our framework delivers substantial gains. As shown in Figure 2, our 4B-parameter model outperforms many 7B model size rerankers, and our 32B model sets a new state-of-the-art on the BRIGHT benchmark. Latency measurements confirm our models maintain the high efficiency of standard pointwise rerankers, making them both powerful and practical.

Briefly, our main contributions are to:

• •

Reveal the suboptimality of binary classification for LLM rerankers and propose a generative approach that outputs discrete integer scores to enhance score discrimination.

• •

Introduce a novel two-stage framework integrating Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to build a single, efficient pointwise reranker for both semantic and reasoning-intensive tasks.

• •

Our model, ERank, sets a new state-of-the-art on the reasoning-based BRIGHT benchmark while demonstrating exceptional performance on standard semantic tasks.

• •

We will open-source our trained models and data to facilitate reproducibility and future research.

2 Related Work

LLM for Text Reranking.

Large Language Models (LLMs) have significantly advanced text reranking beyond the capabilities of earlier encoder-only models such as BERT (Liu et al. 2024). LLMs are typically applied to this task using either zero-shot prompting (Zhang et al. 2023; Zhuang et al. 2024; Niu et al. 2024) or, more effectively, Supervised Fine-Tuning (Ma et al. 2024; Zhang et al. 2024). As shown in Figure 3, reranking methodologies are broadly categorized into pointwise (Liang et al. 2022), pairwise (Qin et al. 2024), and listwise approaches (Ma et al. 2023; Sun et al. 2023). Listwise methods, which evaluate a list of candidate documents, generally yield the highest ranking quality by directly optimizing the document order (Gao, Dai, and Callan 2021; Zhang et al. 2022; Liu et al. 2025). However, their computational cost scales quadratically with input length, making them impractical for real-world systems that demand low latency. In contrast, pointwise methods score each query-document pair independently. This paradigm enables massive parallelization and efficient inference, establishing it as the preferred choice for large scale deployment. Most fine-tuned pointwise rerankers conventionally treat the task as a binary classification problem. We argue this approach fails to leverage the full generative power of modern LLMs and results in suboptimal performance.

Reinforcement Learning for Reranking.

The success of Reinforcement Learning (RL) in enhancing the complex reasoning abilities of LLMs, exemplified by models like OpenAI-O1 (OpenAI 2024) and DeepSeek-R1 (DeepSeek AI 2025), has inspired its application to reranking. Recent work demonstrates that RL can refine a model’s capacity to identify documents that are not merely semantically relevant but also instrumentally useful for resolving a user’s query. However, these pioneering RL-based reranking methods predominantly adopt listwise or setwise training frameworks (Zhuang et al. 2025; Zhang et al. 2025). While effective, they inherit the high latency and memory requirements associated with processing multiple batches of documents sequentially, which limits their practical applicability. Our work addresses this critical gap. We introduce a novel two-stage training pipeline that begins with a generative Supervised Fine-Tuning stage using fine-grained scoring to improve relevance discrimination. Subsequently, we employ RL to optimize our efficient pointwise model with a globally aware listwise reward signal. This strategy achieves the ranking quality of listwise methods while preserving the inference efficiency of a pointwise architecture.

3 Method

Our training methodology unfolds in a two-stage pipeline designed to build a reranker that excels at both semantic relevance and reasoning-intensive relevance. The first stage uses Supervised Fine-Tuning (SFT) to establish a strong foundation, and the second stage employs Reinforcement Learning (RL) with the Group Relative Policy Optimization (GRPO) algorithm (Shao et al. 2024) to refine the reranking ability.

3.1 Task Formulation

We formulate the text reranking task as a generative process. With a specific instruction $I$ that defines the relevance criteria, given a query $q$ and a set of $N$ candidate documents $\{d_{1},d_{2},\dots,d_{N}\}$, our model processes each query-document pair independently. For each pair, it generates a response that includes a Chain-of-Thought (CoT) $c_{i}$ explaining its reasoning, followed by a final relevance score $s_{i}$. This is represented by the conditional probability of the policy LLM $\pi$:

[TABLE]

Based on the extracted scores $\{s_{1},s_{2},\dots,s_{N}\}$, the documents are then sorted in descending order to produce the final ranked list. This pointwise formulation ensures low inference latency and provides interpretability through the generated reasoning Chain-of-Thought (CoT).

3.2 Supervised Fine-Tuning with Fine-Grained Scores

To perform Supervised Fine-Tuning (SFT), we construct a dataset $\mathcal{D}=\{(x_{k},y_{k})\}_{k=1}^{K}$, where each input $x_{k}$ is a prompt containing the relevance instruction $I$, a query $q$, and a document $d$. The target output $y_{k}$ is a combination of reasoning chain $c$ and relevance score $s$, as formatted in Figure 5.

Generative Fine-Grained Scoring.

A central limitation of prior pointwise rerankers is their reliance on a binary classification objective, where the model is trained to predict labels of “yes” and “no” to represent relevance. To better leverage the generative nature of Large Language Models (LLMs), some recent methods have exploited their autoregressive capabilities. These approaches compute a normalized relevance score by extracting token probabilities for “yes” and “no”, which can be definded as:

[TABLE]

Our experiments reveal that such strategy leads to poor score discrimination. The issue becomes particularly pronounced when using reasoning LLMs, since the confidence-boosting effect of Chain-of-Thought (CoT) reasoning causes these models to generate overconfident predictions. As illustrated in Figure 6 and detailed in Appendix B, a comparison between a non-reasoning model (Qwen3-32B (Qwen Team 2025a)) and a reasoning-enhanced model (QwQ-32B (Qwen Team 2025b)) shows that the latter produces normalized scores heavily concentrated near 0 or 1. A significantly higher proportion of scores from the reasoning model falls within the extreme intervals of $[0,0.00001]$ and $[0.99999,1]$. This concentration severely diminishes the model’s ability to distinguish between varying degrees of relevance, which is essential for effective reranking.

To overcome this limitation, we reframe reranking as a generative task with a fine-grained scoring system. Instead of predicting binary labels, we train the model to generate an integer score from [math] to $10$ that reflects the degree of relevance with prompt in Figure 5. Then, the final ranking score is computed as $s_{i}\times\text{Pr(token=}s_{i})$. This method fully utilizes the autoregressive capabilities of the LLM, creating a more expressive and discriminative scoring space critical for distinguishing between documents of varying quality. Table 1 shows that it consistently improves nDCG@10 across multiple benchmarks under experimental settings in Appendix B.

Data Synthesis for SFT.

To train our model for this fine-grained scoring task, we synthesize a high-quality dataset that covers both semantic matching and complex reasoning scenarios. We employ a powerful open-source model, QwQ-32B (Qwen Team 2025b), as a teacher to generate reasoning chains and integer scores. To ensure the quality and reliability of the synthetic labels, our data construction process emphasizes two key aspects. Query-Document Diversity: We source query-document pairs from a diverse mix of datasets, including MS MARCO (Bajaj et al. 2016) for semantic relevance, and ReasonIR (Shao et al. 2025) and Promptriever (Weller et al. 2024) for complex reasoning tasks. For semantic relevance, we randomly select $5,000$ queries from the MS MARCO dataset. For reasoning-intensive tasks, we sample $10,000$ queries from the hard query (HQ) set of ReasonIR and $5,000$ queries from the Promptriever training set. Both of these sources contain complex queries that require deep reasoning. For each query, we enrich the initial candidate pool, which includes the annotated positive documents and synthetic negative documents, by retrieving the top $1,000$ documents from the corpus using the ReasonIR-8B retriever. We then sample documents from different ranking ranges to create a balanced set of negatives: the top $10$ documents serve as hard negatives, positions $11–100$ as medium negatives, and positions $101–1,000$ as easy negatives. Further details are provided in Appendix C. Each query is ultimately associated with exactly $20$ documents to maintain a consistent input structure. High-Quality and Stable Reasoning Trajectory Generation: We use the QwQ-32B teacher model to generate a reasoning chain $c$ and a corresponding score $s$ for each query-document pair. To improve the reliability of these generated labels, we perform multiple independent generations for each instance and compute the average score, which serves as a consensus score. We then select the single generation whose score is closest to this consensus. Experiments on a random sample of $512$ queries show that this strategy significantly improves scoring quality. The average nDCG@10 increased from $63.5\%$ with a single generation to $65.6\%$ with $3$-sample consensus and further to $67.4\%$ with $10$-sample consensus. To balance performance and computational cost, we use three generations per instance in our final data synthesis process. Finally, we filter out any instances where the generated output exceeds $2,048$ tokens. This procedure results in our final Supervised Fine-Tuning (SFT) dataset, $\mathcal{D}$.

Such a high-quality dataset $\mathcal{D}$ enables the model to learn nuanced relevance assessment. The model is then trained on this dataset using a standard language modeling objective:

[TABLE]

3.3 Reinforcement Learning with GRPO

While Supervised Fine-Tuning (SFT) provides a strong reranking model, we employ Reinforcement Learning (RL) to further refine its ability to discern subtle ranking differences and optimize for list level metrics. To achieve this, we adopt the Group Relative Policy Optimization (GRPO) algorithm (Shao et al. 2024), inspired by prior work demonstrating that RL on small, high quality datasets can yield significant performance gains (DeepSeek AI 2025). We initialize both the GRPO policy $\pi_{\theta}$ and the reference model $\pi_{\text{ref}}$ with the SFT-tuned model to ensure training stability and preserve its well generalized capabilities.

The training process begins by sampling a group of $G$ output trajectories $\{y_{1},y_{2},\dots,y_{G}\}$ for each input prompt with the old policy $\pi_{\text{old}}$. The policy $\pi_{\theta}$ is then updated by optimizing the GRPO objective. This objective is built around a clipped importance sampling estimator, which evaluates the advantage of each trajectory relative to others in the group. To prevent the policy from deviating too drastically from the robust SFT model, we incorporate a Kullback Leibler (KL) divergence penalty. This term regularizes the policy updates, ensuring the model learns a more nuanced scoring function without sacrificing its foundational knowledge. The complete objective function is formulated as follows:

[TABLE]

where the clipped estimator $\mathcal{C}$ and KL penalty $D_{\text{KL}}$ are:

[TABLE]

[TABLE]

where the advantage $\hat{A}_{i,t}$ is computed by normalizing the reward $r(y_{i})$ of trajectory $y_{i}$ using the mean and standard deviation of rewards in the group: $\hat{A}_{i,t}=\frac{r(y_{i})-\text{mean}(\{r(y_{j})\}_{j=1}^{G})}{\text{std}(\{r(y_{j})\}_{j=1}^{G})}$. The hyperparameters $\beta$ and $\epsilon$ control the strength of the KL penalty and the clipping range, respectively.

Listwise Reranking Reward Design.

Previous studies have established that listwise rerankers often outperform pointwise models because they directly optimize the relative ordering of documents. Drawing inspiration from this, we designed a novel rule-based listwise reward function, $r_{\text{RR}}$. While our model produces scores pointwise at inference, this reward design allows it to learn from relative document ordering during training, a key strength of listwise methods.

As shown in Figure 7, for a given query, we first generate scores for all $N$ candidate documents and their corresponding $G$ rollouts. These $N\times G$ scores are then aggregated and sorted to determine the global rank of each generated output. Our reward function, $r_{\text{RR}}$, operates based on the following principles. Positive documents receive a high reward based on their reciprocal rank, incentivizing them to be placed as high as possible. Negative documents that are incorrectly ranked higher than any positive document receive a substantial penalty. Negative documents ranked correctly below all positive documents receive a smooth reward based on the squared error against a reference score from the SFT model, which helps maintain a stable scoring distribution. A significant penalty is assigned if the model’s output is not correctly formatted, which discourages generation errors.

The formal definition of the reward $r_{\text{RR}}$ is as follows, where $\phi_{i}^{(j)}$ is the global rank of the $j$-th rollout for document $d_{i}$ with ties resolved by assigning the minimum rank. Let $\mathcal{D}^{P}$ and $\mathcal{D}^{N}$ be the set of positive and negative documents, respectively. Let $\Phi_{\mathcal{D}^{P}}^{(\text{min})}$ and $\Phi_{\mathcal{D}^{P}}^{(\text{max})}$ denote the minimum and maximum ranks for positive documents, respectively.

[TABLE]

where “formatted” condition requires that the model’s output conforms to the expected structure so a score can be extracted. In the third case, $s_{i}$ is the generated score and $t_{i}$ is a reference score from the SFT model $\pi_{\text{ref}}$.

We trained the model using a randomly sampled subset of $2,048$ queries from the SFT dataset, each paired with $20$ documents. The GRPO algorithm was applied directly using these pointwise prompts without any pre-collected trajectories or external reward signals. The SFT tuned model served a dual role as both the initial policy for training and the reference model $\pi_{\text{ref}}$ for calculating KL divergence and providing reference scores for the reward function.

4 Experiment

4.1 Evaluation Setup

Benchmarks.

We evaluate our method across a diverse set of reranking tasks on four benchmark suites. For in-domain semantic matching, we use the TREC DL19 and DL20 passage ranking collections (Craswell et al. 2020, 2021). For out-of-domain generalization, we utilize the entire BEIR benchmark (Thakur et al. 2021) and report results on a five-dataset subset, which we term BEIR-5 (ArguAna, DBPedia, FiQA, NFCorpus, and SCIDOCS), to facilitate efficient ablation studies. To assess complex reasoning, we employ the BRIGHT benchmark (Su et al. 2024) for general reasoning and the FollowIR benchmark (Weller et al. 2025a) for instruction following abilities.

Baselines.

We compare our model, ERank, against leading reasoning-based rerankers. These include JudgeRank-8B, a zero-shot reranker that uses a multi-step reasoning process; Rank1-7B, a pointwise reranker fine-tuned via distillation; Rank-R1-7B, a listwise reranker trained with the GRPO algorithm; and Rearank-7B, a state-of-the-art listwise model trained to predict optimal document permutations. Additionally, we include two top-performing listwise rerankers from the online BRIGHT benchmark (Su et al. 2025): Rank-R1-32B-v0.2 (ielabgroup 2025) trained on the ReasonIR training set, and the zero-shot XRR-Gemini-2.5-Flash (jataware 2025) which performs a two-pass reranking process.

4.2 Implementation and Evaluation Details

For TREC DL, BEIR, and BRIGHT benchmarks, we rerank the top $100$ candidates retrieved by BM25 with Pyserini (Lin et al. 2021). For FollowIR, we rerank the 1,000 candidates provided by the benchmark. Evaluation is performed using nDCG@10 for the first three benchmarks and preference-based Mean Reciprocal Rank ($p$-MRR) for FollowIR. In all cases, higher scores indicate better performance.

Following prior studies (Weller et al. 2025b; Niu et al. 2024; ielabgroup 2025), we adopt similar settings for a thorough evaluation on the challenging BRIGHT benchmark. First, to improve first-stage retrieval precision, we use reasoning queries expanded by GPT-4. Documents are then retrieved using either BM25 or the ReasonIR-8B model (Shao et al. 2025). During the reranking phase, these expanded queries are not provided to the reranker. Second, we employ a hybrid strategy to combine BM25 scores and reranking scores for low-cost model ensembling. While JudgeRank uses a simple weighted sum and Rank-R1-32B-v0.2 adopts min-max normalization, our ERank applies standardization before score aggregation, as detailed in Appendix G.

Using Qwen3 LLM series (Qwen Team 2025a) as the backbone, our ERank model is trained in two-stages. The first stage consists of one epoch of Supervised Fine-Tuning (SFT) with Low-Rank Adaptation (LoRA) (Hu et al. 2022). The second stage uses the GRPO algorithm (Shao et al. 2024) for Reinforcement Learning (RL), performing full-parameter fine-tuning for $10$ epochs with a group $G=5$. Detailed hyperparameters can be found in Appendices E and F. As shown in Table 2, after filtering out those longer than $2,048$ response tokens, there are $14,799$ queries and $2,048$ queries for SFT and RL training, respectively. All experiments are conducted on four NVIDIA A100 (80GB) GPUs. We use official checkpoints for all baselines and reproduce JudgeRank based on its published methodology.

4.3 Main Results

Table 3 presents the main results, with detailed reports available in Appendix H. On average, ERank-4B clearly outperforms all pointwise baselines with 7B or 8B parameters. Furthermore, ERank-4B significantly surpasses listwise rerankers, which are typically more powerful, on reasoning-intensive tasks despite having fewer parameters. This demonstrates that ERank-4B achieves superior effectiveness while maintaining lower latency compared to listwise methods. Beyond the 4B model, we extend our two-stage training pipeline to Qwen3-14B and Qwen3-32B models using identical training data. The results show an overall performance improvement with increased model size, indicating a clear scaling trend. At the 32B scale, our trained ERank-32B reranker outperforms its teacher model, QwQ-32B, which confirms the efficacy of our training procedure.

Table 4 further reports results with advanced retrieval and BM25 hybrid strategy on the BRIGHT benchmark. Our ERank rerankers consistently achieve state-of-the-art performance compared to baselines of similar model size, showing superior robustness and effectiveness. Despite using a pointwise paradigm, ERank-4B achieves a notable nDCG@10 of 38.7 with the BM25 hybrid on documents retrieved by ReasonIR-8B. The ERank-32B model with BM25 hybrid achieves an nDCG@10 of 40.2, outperforming the state-of-the-art Rank-R1-32B-v0.2 listwise reranker. Moreover, ERank-32B approaches the performance of XRR2, a listwise method that employs the Gemini-2.5-Flash model.

4.4 Analysis

Impact of training stages.

To investigate the contribution of SFT and RL to reranking ability, we perform an ablation study using consistent prompts across different model variants. As shown in Table 5, the instructed Qwen3-4B LLM without any fine-tuning performs poorly. Both SFT and RL independently yield significant improvements, highlighting their individual effectiveness. Our two-stage training pipeline for ERank-4B yields the most robust effectiveness overall.

Varying rewards in RL training.

Besides the rule-based listwise reward $r_{\text{RR}}$ using Reciprocal Rank, we evaluate two different rule-based rewards, which are briefly described as follows. Please refer to Appendix I for more details.

• •

Pointwise reward $r_{\text{SE}}$. It uses squared error to measure the difference between the score $s_{i}$ from the policy model and the score $t_{i}$ from the teacher model (i.e., QwQ-32B).

• •

Listwise reward $r_{\text{nDCG}}$. This is a listwise reward similar to $r_{\text{RR}}$, which assesses how effectively positive documents contribute to the nDCG metric while penalizing negative documents ranked above any positive document.

Table 6 compares the results when utilizing different rewards for GRPO training. Overall, listwise rewards such as $r_{\text{nDCG}}$ and $r_{\text{RR}}$ lead to better outcomes than the pointwise reward $r_{\text{SE}}$. Pointwise reward that mimicks the teacher model’s scores for each document independently may not align well with global ranking objectives. In contrast, listwise rewards tend to yield more favorable results by considering relative ranks to encourage a better final reranking order. While $r_{\text{nDCG}}$ shows a notable improvement on the FollowIR benchmark, $r_{\text{RR}}$ demonstrates greater robustness and superior overall performance across the four benchmarks.

Reranking latency.

In real-world applications, achieving superior performance across diverse relevance types must be balanced with acceptable latency. As shown in Figure 8, which measures latency per query on the TREC DL19 dataset, pointwise methods offer significantly lower latency than their listwise counterparts. This advantage stems from the ability of pointwise methods to process documents in parallel, whereas listwise methods require sequential processing as discussed in Section 2. Specifically, the ERank-4B reranker is six times faster than both the listwise methods and the QwQ-32B pointwise reranker, highlighting its practicality for real-world applications. Compared to Rank1-7B, ERank-4B achieves superior performance by generating more tokens while maintaining comparable latency.

5 Conclusion

In this paper, we introduce ERank, an LLM-based reranker designed for effective and efficient reranking of documents in both semantic and reasoning-intensive tasks. To support real-world applications, ERank adopts the pointwise paradigm to ensure low latency while achieving competitive performance through a two-stage training pipeline. The first stage conducts Supervised Fine-Tuning (SFT) to build foundational reasoning capabilities, and the second stage employs the GRPO algorithm with a novel rule-based listwise reward tailored for pointwise rerankers. Extensive evaluation on four benchmarks demonstrates the effectiveness and robustness of ERank compared to state-of-the-art methods.

Appendix A Relevance Types

With examples in Figure 9, we introduce semantic and reasoning-intensive relevance as follows.

Semantic relevance.

It refers to the traditional understanding of relevance based on keyword or semantic matching between a query and a document (Thakur et al. 2021; Craswell et al. 2020, 2021). For example, the query “Do goldfish grow?” can be lexically and semantically matched with “A goldfish will grow to the depth of the water it is kept in.” in the positive document.

Reasoning-intensive relevance.

Different from traditional semantic relevance, reasoning-intensive reranking should be able to capture documents that may not directly answer the query but provide essential intermediate information needed for multi-step reasoning (Su et al. 2024). For example, the query requires that “either four people all know each other or four people are all complete strangers to one another” cannot be directly answered by lexical or semantic matching with the positive document. Instead, the document provides the essential mathematical foundation of Ramsey number $R(n_{1},\dots,n_{c})$, which characterizes the minimum size of a complete graph such that any $c$-coloring of its edges contains a monochromatic complete subgraph of order $n_{i}$. The original query can be transformed into a case of Ramsey number: guests at a party correspond to vertices, mutual acquaintance or stranger status corresponds to a $2$-color edge coloring, and the desired group of $4$ mutual acquaintances or strangers corresponds to a monochromatic $K_{4}$. Thus, solving the query reduces to determining $R(4,4)$. While the answer is not stated explicitly, the document supplies the critical intermediate concept required for multi-step reasoning. Furthermore, queries with external constraints are also reasoning-intensive, determining which types of documents should or should not be considered relevant (Weller et al. 2025a). For example, the query is about the disrupted peace in Ireland and requires that “Any interruptions to the peace process not directly attributable to acts of violence are not relevant.” Here, the positive document is about the violent conflict between two victims from a Northern Ireland outlawed guerrilla group and the Irish Republican Army.

Appendix B Preliminary Experiments

We conduct preliminary experiments on BRIGHT (Su et al. 2024), TREC DL (Craswell et al. 2020, 2021), and BEIR-5 (Thakur et al. 2021) benchmarks in Section 3.2. We use the original queries to retrieve top $100$ candidate documents using the pyserini implementation of BM25 (Lin et al. 2021). Then, we use these original queries and retrieved documents for rereanking under different settings as follows.

Comparing non-reasoning and reasoning LLMs.

For non-reasoning LLM, we use Qwen3-32B and enable its non-reasoning mode to directly output a single word of “yes” or “no”. For reasoning LLM, we use QwQ-32B (Qwen Team 2025b) that outputs Chain-of-Thought (CoT) before giving the binary judgement. The corresponding prompts can be found in Appendix D. After generation, we extract the probability of tokens “yes” and “no” to calculate the normalized probability as discussed in Section 3.2. For each benchmark, we collect the normalized probabilities for all query-document pairs from all subsets to compute the ratios, after which we visualize them in Figures 6 and 10.

Comparing scoring discrimination.

We use the same settings as above, except for using QwQ-32B (Qwen Team 2025b) and varying the scoring discrimination. Specifically, we try scoring discrimination of binary classification, integers from [math] to $3$, integers from [math] to $10$ with the corresponding prompts in Appendix D. After QwQ-32B generation, we extract the output score $s_{i}$ with its probability $\text{Pr}(s_{i})$ for computing the final score as discussed in Section 3.2. Table 1 reports the nDCG@10 averaged on all subsets for each benchmark.

Appendix C Training Dataset

For the hard query dataset (HQ) built by ReasonIR (Shao et al. 2025), we randomly sample $10,000$ queries. In the original dataset, each query has a positive document drawn from the high-quality corpus (Su et al. 2024), and a synthetic hard negative document. To enrich the negative documents, we use the ReasonIR-8B retriever (Shao et al. 2025) to retrieve top $1,000$ negative documents from the same corpus, among which the top $10$ are considered as hard negatives. Also, we randomly sample $4$ documents from positions $11-100$ as medium negatives, and $4$ documents from positions $101-1,000$ as easy negatives. In this way, we obtain $10,000$ queries and each query is associated with $20$ documents, including one positive document and $19$ negative documents.

For the Promptriever training set (Weller et al. 2024), we randomly sample $5,000$ queries, which contain curated instructions that impose additional requirements for relevance judgment. In the original dataset, each query has a positive document drawn from the MS MARCO corpus (Bajaj et al. 2016), and $1$ to $3$ synthetic negative documents. To enrich the negative documents, we use the ReasonIR-8B retriever (Shao et al. 2025) to retrieve top $1,000$ negative documents from the same corpus, among which the top $10$ are considered as hard negatives. For the remaining negative documents, we randomly sample half from positions $11-100$ as medium negatives, and the other half from positions $101-1,000$ as easy negatives. In this way, we obtain $5,000$ queries and each query is associated with $20$ documents, including one positive document and $19$ negative documents.

For the MS MARCO traing dataset (Bajaj et al. 2016), we randomly sample $5,000$ queries, each of which only has one positive documents. Similarly, we use the ReasonIR-8B retriever (Shao et al. 2025) to retrieve top $1,000$ negative documents from the same corpus, among which the top $10$ are considered as hard negatives. For the remaining negative documents, we randomly sample half from positions $11-100$ as medium negatives, and the other half from positions $101-1,000$ as easy negatives. In this way, we obtain $5,000$ queries and each query is associated with $20$ documents, including one positive document and $19$ negative documents.

Finally, we filter out any instances where the generated output exceeds $2,048$ tokens. This procedure results in our final Supervised Fine-Tuning (SFT) dataset $\mathcal{D}$ containing $14,799$ queries, as summarized in Table 2. For RL training, we randomly sample $2,048$ queries from the SFT dataset $\mathcal{D}$.

Appendix D Prompts

For baseline methods, we use prompts from their papers and official repositories, with the corresponding token limits for query and document truncation. For our method and preliminary experiments, queries or documents longer than $2,048$ tokens will be truncated.

For prompts shown in Figure 5 and Figures 11-13, we use different instructions listed in Table 7 for different subsets due to the diverse definitions of relevance, many of which are adapted from ReasonIR paper (Shao et al. 2025). Notably, when generating trajectories for training data in Section 3.2, the instructions used for ReasonIR hard query (HQ) training set (Shao et al. 2025) are actually those used in BRIGHT benchmark. The instruction for MS MARCO training set (Bajaj et al. 2016) is the one for TREC DL benchmark, while the instruction for Promptriever training set (Weller et al. 2024) is the one for FollowIR benchmark.

Appendix E SFT Settings

We use LLaMA-Factory (Zheng et al. 2024) to fine-tune Qwen3-4B-Base, Qwen3-14B-Base, and Qwen3-32B LLMs (Qwen Team 2025a) on NVIDIA A100 (80G) GPUs. We apply LoRA (Hu et al. 2022) on all parameters with rank $32$ and alpha $64$, utilizing the effective batch size of $128$. Detailed parameters are listed in Table 8.

Appendix F RL Settings

We use the GRPO algorithm (Shao et al. 2024) implemented in verl project (Sheng et al. 2024) for training on NVIDIA A100 (80G) GPUs. Detailed detailed configurations are listed in Table 9.

Appendix G Settings on BRIGHT Benchmark

On BRIGHT benchmark, instead of retrieving by BM25 with original queries, existing studies also use different first-stage retrieval and hybridize with BM25 scores to further improve performance (Weller et al. 2025b; Niu et al. 2024; ielabgroup 2025). Thus, we conduct further evaluation with settings described as follows.

First-stage retrieval.

We include the following settings.

• •

Retrieve by BM25 using GPT-4 reason-query. The first-stage top $100$ documents are retrieved by BM25 on GPT-4’s CoT reasoning content. During reranking phase, all rerankers only access the original queries and candidate documents without using such CoTs.

• •

Retrieve by ReasonIR-8B using GPT-4 reason-query. Similarly, the documents are retrieved using the GPT-4’s CoT, except for using the state-of-the-art retriever ReasonIR-8B (Shao et al. 2025). Also, such CoTs are not provided during reranking phase.

BM25 Hybrid.

BM25 hybrid has been widely adopted in recent studies on the BRIGHT benchmark (Niu et al. 2024; Shao et al. 2025; ielabgroup 2025) due to its effectiveness as a low-cost model ensembling strategy. These methods combine a reranking score $s_{i}$ for document $d_{i}$ with its corresponding BM25 score $s_{\text{BM25}}$.

• •

JudgeRank (Niu et al. 2024) calculates a final score as $100\times s_{i}+s_{\text{BM25}}$.

• •

Rank-R1-32B-v0.2 (ielabgroup 2025) first applies min-max normalization to reranking and BM25 scores, respectively. Then, it calculates the final score as $0.1\times$ normalized $s_{\text{BM25}}+0.9\times$ normalized $s_{i}$.

• •

We also apply the same strategy as Rank-R1-32B-v0.2 on ERank rerankers, except that we apply Z-score normalization (standardization) to the scores instead of min-max normalization. After transforming them to have a mean of 0 and a standard deviation of 1, we then calculate the final score as $0.2\times$ normalized $s_{\text{BM25}}+0.8\times$ normalized $s_{i}$.

Appendix H Evaluation Results

Tables 10-13 presents the detailed results of each subset on four benchmarks.

We also present the reward and response length curves for the ERank-4B reranker during GRPO training. As detailed in Appendix F, the training process consists of $10$ epochs, with $32$ steps per epoch. Figure 14 shows that the reward initially increases rapidly. After a brief dip from its peak around step $25$, it grows steadily and saturated after step $160$. In Figure 15, the response length first drops sharply, then gradually increased, saturating at approximately $410$ tokens after $100$ steps.

Appendix I Rule-based Rewards

In Section 4.4, we compare different rule-based reward functions during GRPO training of rerankers.

Pointwise reward $r_{\text{SE}}$.

This reward assesses each query-document pair $(q,d_{i})$ independently as follows.

[TABLE]

where $s_{i}$ is the score given by policy model for pair $(q,d_{i})$, and $t_{i}$ is the corresponding reference score from the teacher model, i.e., QwQ-32B (Qwen Team 2025b) in this paper. “Formatted” indicates that the output follows desired format.

This reward motivates the policy model to generate outputs in the correct format and with scores closely matching the reference scores.

Listwise reward $r_{\text{nDCG}}$.

Similar to $r_{\text{RR}}$ reward discussed in Section 3.3, $r_{\text{nDCG}}$ is also a listwise reward utilizing all $N\times G$ scores from $G$ rollouts for $N$ documents corresponding to a query. For the $j$-th rollout of document $d_{i}$, let $\phi_{i}^{(j)}$ denote its rank among the $N\times G$ scores, with ties resolved by assigning the minimum rank associated with the tied ones.

Denote the positive and negative document sets for query $q$ as $\mathcal{D}^{P}$ and $\mathcal{D}^{N}$, respectively. We first present the Discounted Cumulative Gain (DCG) on $N\times G$ scores:

[TABLE]

where $rel_{i}$ is the relevance score of document $d_{i}$. For simplicity, we assume $rel_{i}=1$ for positive documents ($d_{i}\in\mathcal{D}^{P}$) and $rel_{i}=0$ for negative ones ($d_{i}\in\mathcal{D}^{N}$). Let $\mathbb{I}(d_{i}\in\mathcal{D}^{P})$ be an indicator function that equals $1$ only when $d_{i}\in\mathcal{D}^{P}$, and define $f(\phi_{i}^{(j)})=\frac{1}{\log_{2}\left(\phi_{i}^{(j)}+1\right)}$. We have

[TABLE]

The Ideal Discounted Cumulative Gain (IDCG) is computed by considering the DCG metric when all positive documents are ranked ahead of all negative ones. Using IDCG, the nDCG metric for $N\times G$ scores is calculated as:

[TABLE]

For the $j$-th rollout score of a positive document $d_{i}\in\mathcal{D}^{P}$, its contribution to the overall nDCG metric is $\frac{f(\phi_{i}^{(j)})}{\text{IDCG}}$. Inspired by this observation, we introduce the $r_{\text{nDCG}}$ reward as follows. Let $\Phi_{\mathcal{D}^{P}}=\{\phi_{i}^{(j)}\mid d_{i}\in\mathcal{D}^{P},j\in\{1,2,\cdots,G\}\}$ denote the ranks of all positive documents. We denote the maximum and minimum ranks within this set as $\Phi_{\mathcal{D}^{P}}^{\text{(max)}}$ and $\Phi_{\mathcal{D}^{P}}^{\text{(min)}}$, respectively. The $r_{\text{nDCG}}$ reward is defined as:

[TABLE]

where “formatted” indicates the policy model generates response in desired format and the score $s_{i}^{(j)}$ can be extracted. And $t_{i}$ is the reference score from the reference model $\pi_{\text{ref}}$ (i.e., the SFT-tuned model).