ViSS-R1: Self-Supervised Reinforcement Video Reasoning

TL;DR

This paper introduces ViSS-R1, a self-supervised reinforcement learning framework that enhances multimodal large language models' ability to perform robust, visual-centric video reasoning by integrating pretext tasks into the training pipeline.

Contribution

It proposes the ViSS-R1 framework and Pretext-GRPO algorithm, which improve video reasoning by emphasizing visual information processing and self-supervised learning within the R1 paradigm.

Findings

Outperforms existing methods on six video reasoning benchmarks.

Enhances the model's ability to process transformed visual inputs.

Demonstrates robustness and superiority in complex video reasoning tasks.

Abstract

Complex video reasoning remains a significant challenge for Multimodal Large Language Models (MLLMs), as current R1-based methodologies often prioritize text-centric reasoning derived from text-based and image-based developments. In video tasks, such strategies frequently underutilize rich visual information, leading to potential shortcut learning and increased susceptibility to hallucination. To foster a more robust, visual-centric video understanding, we start by introducing a novel self-supervised reinforcement learning GRPO algorithm (Pretext-GRPO) within the standard R1 pipeline, in which positive rewards are assigned for correctly solving pretext tasks on transformed visual inputs, which makes the model to non-trivially process the visual information. Building on the effectiveness of Pretext-GRPO, we further propose the ViSS-R1 framework, which streamlines and integrates pretext-task-based self-supervised learning directly into the MLLM's R1 post-training paradigm. Instead of relying solely on sparse visual cues, our framework compels models to reason about transformed visual input by simultaneously processing both pretext questions (concerning transformations) and true user queries. This necessitates identifying the applied transformation and reconstructing the original video to formulate accurate final answers. Comprehensive evaluations on six widely-used video reasoning and understanding benchmarks demonstrate the effectiveness and superiority of our Pretext-GRPO and ViSS-R1 for complex video reasoning. Our codes and models will be publicly available.