Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models

TL;DR

This paper introduces a new benchmark and dataset for evaluating and training reward models in video understanding, leading to improved performance and reasoning capabilities.

Contribution

It presents VURB and VUP-35K, the first comprehensive benchmark and large-scale dataset for video reward modeling, along with two high-performing reward models.

Findings

VUP-35K improves reward model performance and reasoning.

VideoDRM and VideoGRM outperform previous models.

Benchmark evaluation shows state-of-the-art results.

Abstract

Multimodal reward models have advanced substantially in text and image domains, yet progress in video understanding reward modeling remains severely limited by the lack of robust evaluation benchmarks and high-quality preference data. To address this, we propose a unified framework spanning benchmark design, data construction, and reward model training. We introduce Video Understanding Reward Bench (VURB), a benchmark featuring 2,100 preference pairs with long chain-of-thought reasoning traces (averaging 1,143 tokens) and majority voting evaluation across general, long, and reasoning-oriented video tasks. We further construct Video Understanding Preference Dataset (VUP-35K) via a fully automated pipeline, providing large-scale high-quality supervision for video reward training. Building on the data, we train VideoDRM and VideoGRM, a discriminative and a generative reward model, both achieving state-of-the-art performance on VURB and VideoRewardBench. Further analysis confirms that VUP-35K enhances both reward performance and model reasoning capability, while VideoDRM and VideoGRM yield significant gains under best-of-$N$ test-time scaling.