Skywork-R1V3 Technical Report

TL;DR

Skywork-R1V3 is a new open-source vision-language model that uses reinforcement learning post-training to significantly improve multimodal reasoning, achieving state-of-the-art results and matching human-level performance.

Contribution

The paper introduces a novel RL-based post-training framework and a reasoning token entropy indicator, enhancing reasoning abilities without additional pre-training.

Findings

Achieves 76.0% on MMMU, matching human capabilities.

Enables smaller models to rival top closed-source VLMs.

Transfers mathematical reasoning to other tasks.

Abstract

We introduce Skywork-R1V3, an advanced, open-source vision-language model (VLM) that pioneers a new approach to visual reasoning. Its key innovation lies in effectively transferring reasoning skills from text-only Large Language Models (LLMs) to visual tasks. The strong performance of Skywork-R1V3 primarily stems from our elaborate post-training RL framework, which effectively activates and enhances the model's reasoning ability, without the need for additional continue pre-training. Through this framework, we further uncover the fundamental role of the connector module in achieving robust cross-modal alignment for multimodal reasoning models. In addition, we introduce a unique indicator of reasoning capability, the entropy of critical reasoning tokens, which has proven highly effective for checkpoint selection during RL training. Skywork-R1V3 achieves state-of-the-art results on MMMU, significantly improving from 64.3% to 76.0%. This performance matches entry-level human capabilities. Remarkably, our RL-powered post-training approach enables even the 38B parameter model to rival top closed-source VLMs. The implementation successfully transfers mathematical reasoning to other subject-related reasoning tasks. We also include an analysis of curriculum learning and reinforcement finetuning strategies, along with a broader discussion on multimodal reasoning. Skywork-R1V3 represents a significant leap in multimodal reasoning, showcasing RL as a powerful engine for advancing open-source VLM capabilities.