VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understanding

TL;DR

VideoRewardBench is a comprehensive new benchmark for evaluating multimodal reward models in video understanding, addressing previous limitations in diversity, scope, and evaluation dimensions, and providing insights into model performance and generalization.

Contribution

We introduce VideoRewardBench, the first extensive benchmark for MRMs in video understanding, including a large dataset and evaluation of 28 models across multiple dimensions.

Findings

Top models achieve only around 57% accuracy.

RL-trained MRMs do not always generalize better.

Inference-time scaling benefits most MRMs except discriminative ones.

Abstract

Multimodal reward models (MRMs) play a crucial role in the training, inference, and evaluation of Large Vision Language Models (LVLMs) by assessing response quality. However, existing benchmarks for evaluating MRMs in the video domain suffer from a limited number and diversity of questions, a lack of comprehensive evaluation dimensions, and inadequate evaluation of diverse types of MRMs. To address these gaps, we introduce VideoRewardBench, the first comprehensive benchmark covering four core aspects of video understanding: perception, knowledge, reasoning, and safety. Through our AI-assisted data pipeline, we curate a high-quality preference dataset of 1,563 annotated samples, including 1,482 unique videos and 1,559 distinct questions--15 times the number found in the most question-rich prior benchmark. Each sample is a triplet consisting of a video-text prompt, a chosen response, and a rejected response. We also conduct a comprehensive evaluation across 28 multimodal reward models spanning three categories: generative, discriminative, and semi-scalar. Results show that even the top-performing model GPT-4o achieves only 57.0% overall accuracy, and the state-of-the-art open-source model Qwen2.5-VL-72B reaches merely 53.3%. Our analysis further reveals three key insights: (i) MRMs trained with reinforcement learning (RL) do not necessarily exhibit stronger cross-modal generalization than those trained without RL; (ii) except for discriminative MRMs, other types of MRMs across varying model capacities can benefit from inference-time scaling; and (iii) variations in input video frame count have different effects on different types of MRMs. We believe VideoRewardBench offers a challenging and valuable benchmark for advancing the evaluation and development of MRMs in the video domain.

Tables

Table 1. Table 1 : Comparison with previous benchmarks on video understanding data. Our VideoRewardBench is the first large-scale and comprehensive benchmark specifically designed to evaluate multimodal reward models in video understanding. The ”#Questions” column indicates the number of distinct video-text prompts across all video understanding preference data.

Dataset	#Questions	Dimension split	Cover holistic dims (e.g., knowledge, safety)	Evaluate all MRM types	Evaluated #MRMs	Highest accuracy
MM-RLHF-RewardBench	10	✗	✗	✗	5	88.0%
JudgeAnything	100	✗	✗	✗	5	–
VideoRewardBench (Ours)	1559	$✓$	$✓$	$✓$	28	63.6%

Table 2. Table 2 : Summary of VideoRewardBench, a comprehensive benchmark covering four major dimensions: perception (including long-form and short-form perception), knowledge, reasoning, and safety. We evaluate both long and short responses. We cover three question types: open-ended QA, multiple choice, and binary judgements.

Dimension	Source	Response Type	Number
Long-Form Perception	VCGBench-Diverse	Long (open-ended generation)	283
283 total			283
Short-Form Perception	MVBench	Short answer (Multiple-choice)	313
	VideoHallucer	Short answer (binary judgment)	100
413 total			413
Knowledge	MMWorld	Long (CoT + answer)	33
	MMVU	Long (CoT + answer)	162
	Video-MMMU	Long (CoT + answer)	43
238 total			238
Reasoning	Video-MME	Long (CoT + answer)	68
	MMBench-Video	Long (CoT + answer)	68
	VSI-Bench	Long (CoT + answer)	142
278 total			278
Safety	Video-SafetyBench	Long (open-ended questions)	351
351 total			351
Grand Total			1563

Table 3. Table 3 : Key Statistics of VideoRewardBench.

Statistic	Number
Total Preference Pairs	1563
- Long-Form Perception	283 (18.1%)
- Short-Form Perception	413 (26.4%)
- Knowledge	238 (15.2%)
- Reasoning	278 (17.8%)
- Safety	351 (22.5%)
Source Datasets (Total)	10
- Long-Form Perception	1
- Short-Form Perception	2
- Knowledge	3
- Reasoning	3
- Safety	1
Total Videos	1482
- Short Videos ($\le 1$ min)	888 (59.9%)
- Medium Videos (1 $\sim$ 5 min)	492 (33.2%)
- Long Videos ($>5$ min)	102 (6.9%)
Question Word Count (avg/max)	28.8/341
Response Word Count (avg/max)	103.8/795
Chosen/Rejected Response Word Count (avg)	102.9/104.6

Table 4. Table 4 : Evaluation of 28 multimodal reward models (MRMs) on VideoRewardBench, covering three categories: generative (with four subtypes), discriminative, and semi-scalar. Models in parentheses are the corresponding base models. The best results are highlighted in bold and the second best is underlined. LLaVA-OV: LLaVA-OneVision. IXC-2.5: InternLM-XComposer-2.5.

Models	#Param	Perception		Knowledge	Reasoning	Safety	Overall Acc	Macro Acc
		long	short
#Samples	–	283	413	238	278	351	1563	1563
Generative Multimodal Reward Models
Proprietary Models (w/o critic training)
GPT-4o-mini (2024-07-18)	–	74.6	47.2	58.8	52.9	44.7	54.4	55.6
GPT-4o (2024-11-20)	–	63.3	50.8	58.8	57.9	57.3	57.0	57.6
Claude-3.7-Sonnet (2025-02-19)	–	65.0	48.4	63.4	58.3	82.9	63.2	63.6
Gemini-2.5-flash (2025-06)	–	61.8	53.0	56.7	49.6	55.0	55.0	55.2
Gemini-2.5-Pro (2025-06)	–	70.7	55.9	65.5	67.3	62.7	63.6	64.4
Open-Source Models (w/o critic training)
Aria	25B	61.8	43.8	51.7	48.9	38.7	48.0	49.0
MiniCPM-o-2.6	8B	59.4	38.7	41.6	34.5	35.6	41.5	42.0
mPLUG-Owl3-7B	7B	57.2	34.6	54.2	50.7	56.7	49.5	50.7
Phi-3.5-Vision	4.2B	61.5	7.75	50.0	52.2	58.4	43.2	46.0
InternVideo2.5-8B	8B	52.3	21.3	39.1	33.1	55.8	39.5	40.3
InternVL3-8B	8B	69.3	48.2	55.0	52.2	64.1	57.3	57.7
InternVL3-78B	78B	70.0	49.2	57.1	50.0	65.8	58.0	58.4
LLaVA-OneVision-7B	7B	59.0	44.1	54.6	58.6	62.7	55.2	55.8
LLaVA-OneVision-72B	72B	64.7	40.9	59.7	53.6	73.5	57.6	58.5
LLaVA-Video-7B	7B	62.5	32.4	44.5	46.0	61.3	48.6	49.4
LLaVA-Video-72B	72B	68.6	41.2	61.8	58.6	70.4	58.9	60.1
Qwen2-VL-72B	72B	68.9	38.0	54.6	52.2	67.0	55.2	56.1
Qwen2.5-VL-7B	7B	56.2	37.5	53.8	46.8	80.1	54.6	54.9
Qwen2.5-VL-72B	72B	68.9	48.4	56.7	52.5	44.7	53.3	54.3
Fast-Thinking Generative MRMs (with critic training)
LLaVA-Critic-7B (LLaVA-OV-7B)	7B	68.2	46.5	50.0	42.1	77.5	57.1	56.9
LLaVA-Critic-72B (LLaVA-OV-72B)	72B	72.4	43.8	55.9	56.5	88.0	63.0	63.3
UnifiedReward (LLaVA-OV-7B)	7B	67.1	48.2	50.4	45.3	71.2	56.6	56.5
Slow-Thinking Generative MRMs (with critic training)
UnifiedReward-Think (LLaVA-OV-7B)	7B	59.7	53.3	50.0	52.9	55.6	54.4	54.3
R1-Reward (Qwen2.5-VL-7B)	7B	36.0	40.0	37.8	30.6	47.9	39.0	38.4
Flex-Judge (Qwen2.5-VL-7B)	7B	35.0	35.1	37.0	37.1	30.2	34.6	34.9
Discriminative Multimodal Reward Models
IXC-2.5-Reward (IXC-2.5)	7B	73.5	51.3	56.3	52.2	38.7	53.4	54.4
Skywork-VL Reward (Qwen2.5-VL-7B)	7B	65.7	49.2	52.9	54.0	80.1	60.5	60.4
Semi-Scalar Multimodal Reward Models
MM-RLHF-Reward (LLaVA-OV-7B)	7B	59.4	37.0	44.1	52.2	65.2	51.2	51.6

Table 5. Table 7 : Details of the multimodal reward models evaluated in our VideoRewardBench. The columns ”Input Frames”, ”Temperature”, and ”Max New Token” denote the parameter settings during inference. The “Input Frames” column shows the default number of input frames, chosen from 2, 4, 8, 16, 32, 64, 128, based on the maximum value that does not exceed the model’s context window and the constraints of GPU memory. ”HF” means ”Hugging Face”. ”–” means that the corresponding model does not have that parameter setting.

Organization	Model	Release	Version	Inference Pipeline	Input Frames	Temperature	Max New Token
Generative Multimodal Reward Models
Proprietary Models (w/o critic training)
OpenAI	GPT-4o-mini	2024-7	gpt-4o-mini-2024-07-18	API	32	0	2048
	GPT-4o	2024-11	gpt-4o-2024-11-20	API	50	0	2048
Anthropic	Claude-3.7-Sonnet	2025-2	claude-3-7-sonnet-20250219	API	20	0	2048
Google	Gemini-2.5-flash	2025-6	Gemini-2.5-flash	API	1fps	0	2048
	Gemini-2.5-Pro	2025-6	Gemini-2.5-Pro	API	1fps	0	2048
Open-Source Models (w/o critic training)
Rhymes	Aria	2024-10	Aria	HF	8	0	2048
OpenBMB	MiniCPM-o-2.6	2025-1	MiniCPM-o-2_6	HF	32	0	2048
mPLUG	mPLUG-Owl3-7B	2024-11	mPLUG-Owl3-7B-241101	HF	64	0	2048
Microsoft	Phi-3.5-Vision	2024-7	Phi-3.5-vision-instruct	HF	8	0	2048
OpenGVLab	InternVideo2.5-8B	2025-1	InternVideo2_5_Chat_8B	HF	128	0	2048
	InternVL3-8B	2025-5	InternVL3-8B	HF	32	0	2048
	InternVL3-78B	2025-5	InternVL3-78B	HF	8	0	2048
lmms-lab	LLaVA-OneVision-7B	2024-10	llava-onevision-qwen2-7b-ov-chat	HF	32	0	2048
	LLaVA-OneVision-72B	2024-10	llava-onevision-qwen2-72b-ov-chat	HF	32	0	2048
	LLaVA-Video-7B	2024-9	LLaVA-Video-7B-Qwen2	HF	64	0	2048
	LLaVA-Video-72B	2024-9	LLaVA-Video-72B-Qwen2	HF	64	0	2048
Alibaba	Qwen2-VL-72B	2024-9	Qwen2-VL-72B-Instruct	HF	64	0	2048
	Qwen2.5-VL-7B	2025-1	Qwen2.5-VL-7B-Instruct	HF	64	0	2048
	Qwen2.5-VL-72B	2025-1	Qwen2.5-VL-72B-Instruct	HF	64	0	2048
Fast-Thinking Generative MRMs (with critic training)
lmms-lab	LLaVA-Critic-7B	2024-10	llava-critic-7b	HF	32	0	4096
	LLaVA-Critic-72B	2024-10	llava-critic-72b	HF	32	0	4096
FDU	UnifiedReward	2025-3	UnifiedReward-7b	HF	32	0	4096
Slow-Thinking Generative MRMs (with critic training)
FDU	UnifiedReward-Think	2025-5	UnifiedReward-Think-7b	HF	32	0	4096
CASIA	R1-Reward	2025-5	R1-Reward	HF	64	1e-06	4096
KAIST AI	Flex-Judge	2025-5	Flex-VL-7B	HF	64	0	4096
Discriminative Multimodal Reward Models
InternLM	IXC-2.5-Reward	2025-1	internlm-xcomposer2d5-7b-reward	HF	16	–	–
Skywork	Skywork-VL Reward	2025-5	Skywork-VL-Reward-7B	HF	64	–	–
Semi-Scalar Multimodal Reward Models
CASIA	MM-RLHF-Reward	2025-2	MM-RLHF-Reward-7B-llava-ov-qwen	HF	32	0	4096

Full text

1 Introduction

Reward models can guide Large Language Models (LLMs) and Large Vision Language Models (LVLMs) to better align with human preferences, and benefit them in training, inference, and evaluation. First, during training, reward models facilitate the generation of high-quality synthetic training data by filtering out low-quality question-answer pairs [10]. Beyond this, they are applicable in post-training stages, such as constructing preference data for Direct Preference Optimization (DPO) [31] and providing reward signals for Reinforcement Learning from Human Feedback (RLHF) [27, 33, 47]. Although rule-based rewards can replace reward models in logical reasoning and factual judgment tasks, reward models are still required in domains where verifiable answers are unavailable, such as open-domain question answering and safety-related tasks. Second, at inference time, they can be integrated with test-time scaling strategies to enhance performance by selecting the best response from multiple candidates [32, 47, 35]. Third, during evaluation, high-quality reward models can offer reliable and scalable assessment, thereby promoting the advancement of foundation models.

Despite the critical role of reward models in the development of LLMs and LVLMs, existing benchmarks for evaluating reward models are typically limited to text and image modalities. Recent work [50, 30] has evaluated multimodal reward models (MRMs) in the video domain. However, as shown in Table 1, these efforts face three major limitations. (1) Limited number and diversity of questions: The number of distinct video-related questions in these benchmarks is at most 100, and they are primarily restricted to open-ended general video QA tasks, lacking coverage of different question types. (2) Lack of comprehensive evaluation dimensions: These benchmarks do not adopt any dimensional breakdown and are primarily limited to the perception domain, lacking coverage of knowledge, safety, and reasoning (e.g., spatial reasoning). (3) Insufficient analysis of existing MRM categories: Prior works evaluate only a limited number of MRMs, primarily focusing on generative MRMs. Moreover, they do not investigate how inference-time scaling or video frame sampling rates affect MRM performance. To bridge these gaps, we introduce VideoRewardBench, the first comprehensive benchmark for evaluating multimodal reward models in the video domain across four key dimensions: perception, knowledge, reasoning, and safety. The perception dimension is divided into two sub-dimensions based on different question types: long-form perception and short-form perception. To avoid data contamination, we exclude any publicly available video preference data used for training. We collect video-text pairs from multiple open-source benchmarks and apply an AI-assisted, multi-stage filtering process to ensure question quality and difficulty. Then, multiple LVLMs are used to generate candidate responses, followed by human annotation to determine the chosen and rejected response within each preference pair. We retain only the preference pairs with sufficient inter-annotator agreement, resulting in a total of 1,563 triplets of (prompt, chosen response, rejected response). Then, the MRMs are tasked with identifying which of the two responses is better. In total, VideoRewardBench includes 1,559 different prompts—more than 15 times the number in previous benchmarks.

We also conduct comprehensive evaluation of 28 multimodal reward models (MRMs), convering generative, discriminative and semi-scalar. Our VideoRewardBench reveals significant limitations of existing MRMs: leading proprietary models like Gemini-2.5-Pro and Claude-3.7-Sonnet achieve only moderate performance (Gemini-2.5-Pro: 63.6%, Claude-3.7-Sonnet: 63.2%), while GPT-4o performs at just 57.0%. The top-performing open-source LVLM Qwen2.5-VL-72B achieves only 53.3% overall accuracy. All open-source MRMs still lag behind the best proprietary model. Most models struggle in short-form perception, knowledge, and reasoning. We also examine the impact of inference-time scaling and video frame count on different categories of MRMs. Our experiments uncover three key findings: (1) MRMs trained with reinforcement learning (RL) do not necessarily exhibit stronger cross-modal generalization than those trained without RL, such as generative MRMs trained via supervised fine-tuning (SFT) and top-performing discriminative MRMs. (2) Unlike the findings in the VL-RewardBench, under proper settings and aggregation methods, all types of MRMs except discriminative ones consistently benefit from inference-time scaling across different model capacities. (3) Increasing the number of sampled frames generally improves performance on most traditional video benchmarks. However, frame count variation affects different categories of MRMs differently. As the frame count increases, critic-trained generative MRMs benefit more than other categories, whereas semi-scalar MRMs show a slight decline. We will release all data and evaluation code upon acceptance.

Our main contributions are summarized as follows:

• We propose the first large-scale and comprehensive benchmark for evaluating multimodal reward models (MRMs) in the video domain across four holistic evaluation dimensions: perception, knowledge, reasoning, and safety.

• We conduct a thorough analysis of all existing types of MRMs: generative, discriminative, and semi-scalar.

• We investigate the impact of inference-time scaling and different sampled frame count on different MRM types.

2 Related Works

2.1 Multimodal Reward Models

Existing multimodal reward models (MRMs) can be classified as three categories based on their reward generation paradigms: discriminative (scalar), semi-scalar, and generative. For the discriminative MRMs, they directly assigns a scalar score for the given prompt and response. Works like IXC-2.5-Reward [47] and Skywork-VL Reward [36] construct a large amount of high-quality preference data for training. For the semi-scalar MRMs [50], they first generate a textual critique of the evaluated response, and then predict a scalar reward value based on the prompt, the response, and the generated critique. For the generative MRMs, this method directly employs LVLMs as MRMs that only generate textual critiques for the given responses. Early studies [42, 5, 40] construct critic instruction-following datasets and primarily utilize supervised fine-tuning (SFT) during training. Recent research [49, 39, 29, 52] has increasingly focused on leveraging reinforcement learning (e.g., GRPO) to train generative MRMs. To systematically analyze existing MRMs, our benchmark thoroughly evaluates three categories of MRMs.

2.2 Reward Models Benchmarking

Existing benchmarks for evaluating reward models are mainly limited to text and image modalities. Works like RewardBench [14], RM-Bench [22], ReWordBench [41], and REWARDBENCH 2 [24] evaluate reward models in purely textual settings. Works, such as MLLM-as-a-Judge [4], VL-RewardBench [19], and Multimodal RewardBench [45], assess multimodal reward models (MRMs) in image understanding. Although recent works (e.g., MM-RLHF-RewardBench [50], JudgeAnything [30]) have evaluated MRMs in video understanding, they suffer from limited questions and fail to cover important areas such as knowledge and safety. In contrast, our VideoRewardBench is the first large-scale and holistic benchmark that evaluates MRMs in video understanding, covering four key dimensions.

3 VideoRewardBench

3.1 Overview

Similar to previous works [14, 19, 45], each sample in VideoRewardBench is a preference pair $(x,y_{c},y_{r})$, where $x$ denotes a prompt consisting of a video and a textual question, $y_{c}$ is the chosen response, and $y_{r}$ is the rejected response. Table 2 summarizes our VideoRewardBench. Following prior comprehensive evaluations of LLMs [20] and LVLMs [15], we focus on assessing multimodal reward models across four key dimensions: perception, knowledge, reasoning, and safety. The perception dimension is divided into two major subcategories: long-form perception and short-form perception. In the following subsections, we present the construction process of VideoRewardBench in three stages: prompt collection, response collection, and preference annotation. Finally, we present comprehensive dataset statistics.

3.2 Prompt Collection

3.2.1 Dataset Source

Perception. This dimension assesses the ability of multimodal reward models to evaluate responses to perception-oriented questions. It is divided into two subsets based on question types: long-form perception and short-form perception. Long-form perception focuses on open-ended long-form generation tasks. We collect 400 prompts from VCGBench-Diverse [23] by uniformly sampling from the three tasks: dense video caption, spatial understanding, and temporal understanding. In contrast, short-form perception targets various short-form video QA tasks, including spatiotemporal perception and hallucination detection. We draw all spatiotemporal perception-related prompts from MVBench [18], a comprehensive video understanding benchmark covering diverse temporal skills. For hallucination detection, we select all prompts from VideoHallucer [37], a thorough benchmark for detecting video hallucinations. In both datasets, the prompts are presented as multiple-choice or binary (yes/no) formats.

Knowledge. This dimension covers tasks that demand specialized knowledge in various domain such as science, engineering, medicine, business, and humanities. We source knowledge-related prompts from three multi-discipline datasets: MMWorld [11], MMVU [51], and Video-MMMU [12]. We adopt the human-annotated subset of MMWorld and all prompts in MMVU for later processing. Video-MMMU evaluates LVLMs’ knowledge acquisition abilities in three stages: perceiving information, comprehending knowledge, and adapting knowledge. To ensure difficulty, we use only the subset related to comprehending knowledge.

Reasoning. This dimension focuses on four aspects of reasoning abilities across spatial reasoning, temporal reasoning, relation reasoning, object reasoning. We collects prompts from three datasets: Video-MME [7], MMBench-Video [6] and VSI-Bench [43]. For Video-MME, we select its reasoning subset including four tasks: temporal reasoning, spatial reasoning, action reasoning, object reasoning. For MMBench-Video, we utilize prompts from the temporal reasoning, relation reasoning, and attribute reasoning subsets. For VSI-Bench, which evaluates LVLMs’ visual-spatial intelligence, we select prompts from six tasks identified in VSI-Bench’s error analysis as exhibiting prominent spatial reasoning errors: object size, relative distance, absolute distance, room size, relative direction, and route plan.

Safety. This dimension is designed to evaluate the safety awareness of multimodal reward models. We aim to encompass a diverse range of safety categories. Video-SafetyBench [21] provides the first comprehensive evaluation for video-text multimodal attacks across 13 primary unsafe categories and 48 fine-grained sub-categories. We select all 1,132 prompts that contain benign textual queries with higher attack success rates in Video-SafetyBench for later processing.

3.2.2 Multi-Stage Filtering

To ensure prompt quality, we employ a multi-stage filtering to remove questions answerable without video content or those that are too easy or too difficult. For the short-form perception, knowledge, and reasoning dimensions, where most questions are non-open-ended, we first discard prompts with video longer than 10 minutes. Then, we prompt a strong model to answer the textual questions without providing any video input, and remove those that it answers correctly. In the final stage, we input each prompt (video-text query) into Qwen2-VL-7B-Instruct [34] and discard those prompts that it can answer correctly. For the long-form perception and safety dimensions, where the questions are open-ended, we only filter prompts with video durations exceeding 10 minutes in this step.

3.3 Response Collection and Preference Annotation

3.3.1 Long-form Perception.

We sample responses using the following five models: LLaVA-Video-72B, Qwen2.5-VL-72B, and three proprietary models. For each prompt, we randomly select three models to generate responses. These responses are then paired to form three preference pairs. During human annotation, three different annotators are asked not only to judge which response is better or whether it is a tie, but also to assess the preference strength. We take the result of majority voting as the final annotation and discard preference pairs with no consensus or a tie. For each remaining pair, we compute the average preference strength, where a higher value indicates greater distinguishability between responses. If a prompt corresponds to multiple preference pairs, we discard the one with the highest average strength and randomly sample one from the rest.

3.3.2 Short-form Perception.

In this dimension, the responses under evaluation are concise, presented as either multiple-choice options or binary (yes/no) answers. Each prompt is associated with a ground-truth answer, which is used as the chosen response. An incorrect option or the opposite judgment is then generated as the rejected response. Hence, human annotations are not used for this dimension.

3.3.3 Knowledge & Reason.

Given the higher difficulty in these two dimensions, we sample responses from three leading proprietary models. Moreover, these two dimensions primarily emphasize the response correctness rather than stylistic differences. Therefore, following Multimodal RewardBench [45], we randomly select one model from the above list to generate 10 responses. By comparing the ground-truth answer with the final answer of each response, we filter out prompts where the model answers all 10 instances either correctly or incorrectly, thereby removing questions that are either too simple or too difficult. These cases fail to effectively differentiate model capabilities [28]. For each remaining prompt, we select two responses from its 10 responses to construct an initial preference pair: one with correct final answer and the other with incorrect final answer. Then, for each preference pair, three different annotators examine the intermediate reasoning process of the response with the correct final answer. If any annotator identifies a critical flaw, the preference pair is discarded.

3.3.4 Safety.

We adopt the five models used in the long-form perception dimension and add another closed-source model with strong safety performance. For each prompt, we sample six responses using these six models. Each response is evaluated using the RJScore, a safety metric proposed in Video-SafetyBench [21], to assign it to either the successfully attacked response set or the unsuccessfully attacked response set. We remove the prompts whose six responses are either all successfully attacked or all unsuccessfully attacked. To further ensure prompt quality and difficulty, we retain only the prompts with an attack success rate greater than 50% across the six responses. For each remaining prompt, we select one response from its successfully attacked response set and one from the unsuccessfully attacked set with the closest lengths to form an initial preference pair. Each pair is then annotated by three different annotators, with the final label determined by majority voting. We retain only pairs where the chosen response is not successfully attacked, while the rejected one is.

3.4 Dataset Statistics

A summary of our VideoRewardBench is presented in Table 2, and its key statistical information is shown in Table 3. As shown in Table 3, VideoRewardBench contains 1,563 preference pairs sourced from 10 existing video benchmarks. Moreover, our preference data achieves approximate balance across five different dimensions, ensuring task balance. The dataset includes 1,482 unique videos, with durations ranging from less than one minute to 10 minutes.

To analyze potential length bias, we computed the average word count of the chosen and rejected responses across all preference data, which are 102.9 and 104.6, respectively. These similar values suggest minimal bias in response length. Additionally, in Figure 2, we analyze the distribution of the word count difference between the chosen and rejected responses. The x-axis represents the word count difference, computed as: $\frac{l_{rejected}-l_{chosen}}{l_{chosen}}\times 100\%$, where $l_{chosen}$ and $l_{rejected}$ denote the word counts of the chosen and rejected responses, respectively. The y-axis indicates the number of preference pairs corresponding to each difference value. The resulting bell-shaped distribution centered around zero indicates that our preference labels are primarily determined by response quality instead of response length.

Detailed examples for each evaluation dimension can be found in the Appendix.

4 Experiments

4.1 Evaluation Models

We conduct a thorough evaluation across all major categories: generative, discriminative, and semi-scalar MRMs. For generative MRMs, we assess both non-critic-trained and critic-trained models. The non-critic-trained generative MRMs include proprietary models such as GPT-4o [25], GPT-4o-mini [26], Claude-3.7-Sonnet [2], Gemini-2.5-Pro [9], and Gemini-2.5-Flash [8], as well as open-source models of varying sizes, including Aria (25B) [17], MiniCPM-o-2.6 (8B) [44], mPLUG-Owl3-7B [46], Phi-3.5-Vision (4.2B) [1], InternVideo2.5-8B [38], InternVL3 (8B, 78B) [53], LLaVA-OneVision (7B, 72B) [16], LLaVA-Video (7B, 72B) [48], Qwen2-VL-72B [34], and Qwen2.5-VL (7B, 72B) [3]. The critic-trained generative MRMs are divided into fast-thinking MRMs such as LLaVA-Critic (7B,72B) [42] and UnifiedReward [40], and slow-thinking MRMs such as R1-Reward [49], UnifiedReward-Think [39], and Flex-Judge [13]. For discriminative MRMs, we evaluate IXC-2.5-Reward [47] and Skywork-VL Reward [36]. As for semi-scalar MRMs, we evaluate MM-RLHF-Reward [50].

4.2 Implementation Details

4.2.1 Generative MRMs

All generative MRMs are evaluated via a pairwise ranking approach. Given a video-text query, two candidate responses, and a prompt template, each MRM is tasked with selecting the better response. To prevent order bias, the order of the two responses in each preference pair in our benchmark is randomly shuffled. For non-critic-trained generative MRMs, we employ a prompt template similar to the LLM-as-a-judge template proposed in RewardBench [14]. Our detailed prompt template is in the Appendix. For critic-trained generative MRMs, we utilize their pairwise ranking prompt templates and official inference code to ensure optimal performance.

4.2.2 Discriminative & Semi-Scalar MRMs

To evaluate discriminative and semi-scalar MRMs, we adopt a pointwise scoring strategy. Each benchmark sample (video-text prompt $x$, chosen response $y_{c}$, rejected response $y_{r}$) is split into two tuples: ($x$, $y_{c}$) and ($x$, $y_{r}$). These two tuples are then separately fed into the two types of MRMs to obtain scores for the chosen and rejected responses, respectively. If the chosen response receives a higher score, the model’s judgment is considered to be aligned with human preference.

4.2.3 Evaluation Metric and Parameter

Following VL-RewardBench [19], we report accuracy for each evaluation dimension, along with two overall metrics: Overall Accuracy, which measures the proportion of samples where the model’s judgment matches human preference, and Macro Average Accuracy, calculated as the mean accuracy across five dimensions: long-form perception, short-form perception, knowledge, reasoning, and safety.

For generative and semi-scalar MRMs, the temperature is set to 0 and the maximum output length defaults to 2048. The input frame count for each model and other parameter settings during inference are provided in the Appendix.

4.3 Evaluation Results

Table 4 shows the thorough evaluation results on VideoRewardBench. Our key findings are summarized as follows:

VideoRewardBench poses significant challenges and reveals a clear performance gap among existing MRMs. Gemini-2.5-Pro performs best with an overall accuracy of 63.6%, followed by Claude-3.7-Sonnet at 63.2%, while GPT-4o reaches only 57%. Among open models, LLaVA-Critic-72B leads with 63%, whereas the top-performing open-source LVLM Qwen2.5-VL-72B attains 53.3%. Notably, critic-trained generative, discriminative, and semi-scalar MRMs have undergone specialized reward modeling training on image or video data. However, all of these models still fall short of proprietary models like Gemini-2.5-Pro. Among the 28 evaluated MRMs, only four—Gemini-2.5-Pro, Claude-3.7-Sonnet, LLaVA-Critic-72B, and Skywork-VL Reward—achieve overall accuracies exceeding 60%. Moreover, VideoRewardBench reveals a clear performance gap, with top models near 63% accuracy and the lowest at 34.6%, a nearly 30-point difference.

Most models struggle across the dimensions of short-form perception, knowledge, and reasoning. For short-form perception, most open-source models achieve accuracies below 50%, and Gemini-2.5-Pro only reaches 55.9%. This indicates that our multi-stage filtering process during prompt collection effectively selects challenging samples. Existing MRMs exhibit substantial deficiencies in handling difficult short-form video QA tasks. For the knowledge and reasoning dimensions, most open models fail to surpass 60% accuracy. This suggests that these models—even some slow-thinking critic-trained MRMs—may lack the necessary knowledge and complex reasoning capabilities in the video domain.

Scaling up model size generally improves performance across both open and proprietary models. For open models, different model families—except Qwen2.5-VL—exhibit overall performance improvements when scaling up model size. For instance, LLaVA-Video improves from 48.6% to 58.9% (7B to 72B), and LLaVA-Critic from 57.1% to 63.0%. However, the extent of performance gains varies across model families. The critic-trained MRM LLaVA-Critic benefits more from model scaling than its non-critic-trained counterpart, LLaVA-OV. In contrast, scaling from Qwen2.5-VL-7B to Qwen2.5-VL-72B leads to a slight drop in overall accuracy (from 54.4% to 53.3%) primarily due to a substantial decrease on the safety dimension (from 80.1% to 44.7%), despite consistent improvements in all other dimensions. Model scaling also benefits proprietary models: GPT-4o (57%) outperforms GPT-4o-mini (54.4%).

MRMs trained with reinforcement learning (RL) do not necessarily exhibit stronger cross-modal generalization than those trained without RL. Among critic-trained generative MRMs, fast-thinking MRMs including LLaVA-Critic and UnifiedReward are trained via supervised fine-tuning (SFT) on fast-thinking critic instruction-following datasets, based on LLaVA-OV. For slow-thinking MRMs, UnifiedReward-Think and R1-Reward are trained through a SFT+RL pipeline, based on LLaVA-OV-7B and Qwen2.5-VL-7B respectively. Flex-Judge is trained on textual slow-thinking critic data based on Qwen2.5-VL-7B. Except for UnifiedReward and UnifiedReward-Think, which incorporate video-related data during training, the remaining models are trained solely on image or text-based datasets. On VideoRewardBench, fast-thinking generative MRMs show consistent improvements over their base models. For example, LLaVA-Critic-7B outperforms LLaVA-OV-7B by 1.9%, and LLaVA-Critic-72B surpasses LLaVA-OV-72B by 5.4%. In contrast, MRMs trained via RL show varying degrees of performance degradation compared to their base models. UnifiedReward-Think drops by 0.8% relative to LLaVA-OV-7B, and R1-Reward drops by 15.6% compared to Qwen2.5-VL-7B. Although R1-Reward outperforms the fast-thinking MRMs on image benchmarks, it exhibits weaker cross-modal generalization when transitioning from image to video modalities. Similarly, Flex-Judge, trained only on text and evaluated on video, shows the largest drop—20% compared to its base model. Additionally, the top-performing discriminative MRM, Skywork-VL Reward, also shows stronger cross-modal generalization.

5 Analysis

5.1 Inference-time Scaling

Given the failure of inference-time scaling in improving non-critic-trained generative MRMs for image understanding [19], we further investigate its effect on a broader range of MRMs in video understanding. For each test sample, we sample K responses ($K\in[1,9]$) from the MRMs with temperature 1.0. For generative MRMs, to minimize positional bias, we reverse the order of the two candidate responses in half of the K samplings per sample, leaving the others unchanged. We aggregate the results via majority voting. For discriminative MRMs, as these MRMs output deterministic scores, inference-time scaling offers no performance gain. For semi-scalar MRMs, we compare two aggregation methods: majority voting and score merging. In score merging, we compute the final score for each of the two candidate responses by summing the scores obtained from K independent inference runs, and use these aggregated scores for comparison.

As shown in Figure 3, we observe three key findings: (1) Except for discriminative MRMs, inference-time scaling improves performance for both generative and semi-scalar MRMs. For example, Claude-3.7-Sonnet improves by 10.6% as K increases from 1 to 9, and Qwen2.5-VL-72B improves by 2% as K rises from 1 to 5. Although the semi-scalar MRM MM-RLHF-Reward fails to achieve performance gains with majority voting, it improves from 54.5% to 56.5% using score merging as K increases from 1 to 7. (2) MRMs trained via RL, such as R1-Reward, benefit significantly more from inference-time scaling (14.3% gain from K=1 to 9) than the non-critic-trained base model Qwen2.5-VL-7B (5.5%) or fast-thinking MRMs like LLaVA-Critic-7B (6.7%). (3) Within the same model family, larger models do not necessarily gain more than smaller ones. For example, Qwen2.5-VL-7B improves by 5.5%, while Qwen2.5-VL-72B gains only 0.9%. Additionally, in the work VL-RewardBench [19] that evaluates multimodal reward models in image understanding, the failure during their inference-time scaling was mainly due to their inappropriate choice of sampling parameters: temperature = 0.2 and top-p = 0.2. In contrast, setting the temperature to 1.0 is appropriate for most multimodal reward models.

We further explore the upper bounds of inference-time scaling for both critic-trained and non-critic-trained MRMs (Figure 4(a)) and examine the effects of different temperature settings on LLaVA-Critic-7B/72B (Figure 4(b)). In Figure 4(a), we compare R1-Reward, trained via reinforcement learning, with its non-critic-trained base model Qwen2.5-VL-7B. As we increase the number of samples up to 21, we observe that although R1-Reward shows a significantly larger improvement than Qwen2.5-VL-7B, both models eventually converge to a similar upper bound in inference-time scaling. In Figure 4(b), we compare two temperature settings, 0.5 and 1.0, for LLaVA-Critic-7B and LLaVA-Critic-72B. Both models generally exhibit an upward trend as K increases across different settings. In particular, LLaVA-Critic-72B consistently underperforms greedy decoding when the temperature is set to 1.0. However, when the temperature is set to 0.5 and K = 9, its performance surpasses that of greedy decoding. It is evident that, for the LLaVA-Critic series, setting the temperature to 0.5 yields substantially better performance than 1.0. These results also suggest that to outperform greedy decoding through inference-time scaling, we can consider adjusting the temperature parameter or increasing the number of samples.

5.2 Comparison of Different Frame Counts

Given that increasing sampled frames generally improves performance on traditional video benchmarks, we study how varying input frame counts affect different types of MRMs. We evaluate six typical MRMs covering all major categories, varying sampled frames from 1 to 64 with a fixed sampling temperature of 0. In Figure 5, increasing input frames affects different types of MRMs differently. Not all MRMs exhibit performance gains as more frames are provided. For generative MRMs, critic-trained generative MRMs show a clear upward trend. For instance, LLaVA-Critic-72B improves from 52.0% to 63.0% as the frame count rises from 1 to 64. In contrast, non-critic-trained generative MRMs (Qwen2.5-VL-7B/72B) exhibit a relatively less pronounced upward trend, likely due to their limited evaluation ability and challenges with large visual inputs. Notably, for Qwen2.5-VL, we observe a noticeable performance drop on safety tasks as the frame count rises. The accuracy of Qwen2.5-VL-72B in the safety dimension drops by 6% as the frame count rises from 2 to 64, whereas critic-trained generative MRMs exhibit improved safety evaluation with more frames. For the discriminative MRM, Skywork-VL Reward exhibits substantial performance fluctuations at low frame counts but stabilizes as frames increase. For the semi-scalar MRM, MM-RLHF-Reward is the least affected by frame count variation, showing a slight initial drop before stabilizing.

6 Conclusions

We present VideoRewardBench, the first large-scale and comprehensive benchmark specifically designed to evaluate multimodal reward models (MRMs) across four key dimensions in video understanding. Through a systematic evaluation of 28 MRMs spanning all major categories, we reveal their significant limitations in video understanding evaluation. VideoRewardBench offers a challenging testbed for MRMs in this domain. Moreover, we study the effects of inference-time scaling and varying the sampled frame count on different types of MRMs. Our findings provide valuable insights for advancing MRMs in video understanding.

Appendix A Evaluation Details

A.1 Evaluation Prompts

To evaluate non-critic-trained generative multimodal reward models (MRMs) via pairwise ranking, we utilize the prompt template in Table 5. For critic-trained generative MRMs, we utilize their official pairwise ranking prompt templates.

A.2 Evaluation Result Extraction

For all generative MRMs, we first extract their judgment results from their outputs using code. If this fails, we input the MRM’s output and the prompt template in Table 6 into a proprietary model to extract the judgment result. For the discriminative and semi-scalar MRMs, since these models directly output scores for each response, we obtain their judgment results by comparing the scores assigned to the two responses in each preference pair.

A.3 Evaluation Parameters

Table 7 presents the detailed information and inference parameters of all evaluated multimodal reward models. We use each model’s official implementation to process visual inputs. For models that support video input, such as Qwen2.5-VL and Gemini, we directly feed the video into the model for evaluation. For models that do not support video input, such as GPT-4o, we uniformly sample multiple frames from the video and use them as input instead.

Appendix B Source Datasets Introduction

In this subsection, we introduce the 10 source video benchmarks used in our VideoRewardBench.

• VCGBench-Diverse: This benchmark contains 4,354 question-answer pairs designed to evaluate LVLMs across dense video captioning, spatial and temporal understanding, and complex reasoning, spanning 18 broad video categories.

• MVBench: By converting various static spatial understanding tasks into dynamic ones, this dataset covers 20 video tasks that require temporal understanding. It serves as a comprehensive benchmark for video understanding, encompassing a wide range of temporal skills, from perception to cognition.

• VideoHallucer: This dataset is a comprehensive benchmark for video hallucination detection, focusing on two types of hallucinations: intrinsic and extrinsic. Intrinsic hallucinations are classified into object-relation, temporal, and semantic detail, while extrinsic hallucinations include factual hallucination and non-factual hallucination. The dataset comprises a total of 1,800 questions and 948 videos.

• MMWorld: This benchmark provides multi-discipline and multi-faceted reasoning evaluation, covering seven broad disciplines and 69 subdisciplines. The seven broad disciplines include Art & Sports, Business, Science, Health & Medicine, Embodied Tasks, Tech & Engineering, and Games. This dataset features multi-faceted reasoning, covering explanation, counterfactual thinking, and future prediction. In total, it contains 6,627 questions and 1,910 videos.

• MMVU: MMVU is a comprehensive expert-level, multi-discipline benchmark, spanning four core disciplines and 27 subjects. The four primary disciplines are Science, Healthcare, Humanities & Social Sciences, and Engineering. This dataset contains a total of 3,000 questions and 1,529 distinct videos.

• Video-MMMU: This benchmark provides knowledge-related prompts covering six disciplines for evaluating LVLMs’ knowledge acquisition abilities in three stages: perceiving information, comprehending knowledge, and adapting knowledge. This dataset contains 900 questions and 300 videos, covering six major disciplines: Art, Business, Science, Medicine, Humanities, and Engineering.

• Video-MME: Video-MME provides a full-spectrum evaluation of video analysis by focusing on three main tasks: perception, reasoning, and information synopsis.

• MMBench-Video: This holistic video understanding benchmark features a three-level hierarchical capability taxonomy, with the top level comprising two primary capabilities: perception and reasoning. The perception category is divided into fine-grained perception, coarse perception, and hallucination, while the reasoning category includes temporal reasoning, commonsense reasoning, relation reasoning, attribute reasoning, and logic reasoning. At the third level, the taxonomy comprises 26 fine-grained capability categories.

• VSI-Bench: It evaluates LVLMs’ visual-spatial intelligence across eight tasks: object count, object size, relative distance, absolute distance, appearance order, room size, relative direction, and route plan. This dataset contains over 5,000 questions and 288 videos.

• Video-SafetyBench: This benchmark provides the first comprehensive evaluation for video-text multimodal attacks. It contains 2,264 video-text pairs, covering 13 primary unsafe categories and 48 fine-grained sub-categories. The 13 primary categories include Violent Crimes, Non-Violent Crimes, Sex-Related Crimes, Child Sexual Exploitation, Defamation, Specialized Advice, Privacy, Intellectual Property, Indiscriminate Weapons, Hate, Suicide & Self-Harm, Sexual Content, and Elections. Each video is associated with two queries: a harmful query, which explicitly contains harmful content at the textual level, and a benign query, which appears harmless in text but conveys harmful information when combined with the video.

Appendix C Case Study

As shown in Figures 6–10, we sequentially present representative samples from the five evaluation dimensions: long-form perception, short-form perception, knowledge, reasoning, and safety.