Video-Based Reward Modeling for Computer-Use Agents

TL;DR

This paper introduces a scalable, video-based reward model for evaluating computer-use agents, leveraging a new dataset and novel techniques to accurately assess task success from execution videos.

Contribution

The paper presents ExeVR-53k, a large dataset of execution videos with rewards, and a new reward model that predicts task success from videos, outperforming proprietary models.

Findings

ExeVR-53k dataset with 53k video-reward triplets

ExeVRM achieves 84.7% accuracy in success prediction

Model outperforms GPT-5.2 and Gemini-3 Pro in evaluations

Abstract

Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction. In this work, we study reward modeling from execution video: a sequence of keyframes from an agent trajectory that is independent of the agent's internal reasoning or actions. Although video-execution modeling is method-agnostic, it presents key challenges, including highly redundant layouts and subtle, localized cues that determine success. We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets. We further propose adversarial instruction translation to synthesize negative samples with step-level annotations. To enable learning from long, high-resolution execution videos, we design spatiotemporal token pruning, which removes homogeneous regions and persistent tokens while preserving decisive UI changes. Building on these components, we fine-tune an Execution Video Reward Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success. Our ExeVRM 8B achieves 84.7% accuracy and 87.7% recall on video-execution assessment, outperforming strong proprietary models such as GPT-5.2 and Gemini-3 Pro across Ubuntu, macOS, Windows, and Android, while providing more precise temporal attribution. These results show that video-execution reward modeling can serve as a scalable, model-agnostic evaluator for CUAs.