Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning

TL;DR

Metis-RISE introduces a novel training paradigm for multimodal reasoning models that combines reinforcement learning without initial supervised fine-tuning, followed by targeted SFT, resulting in state-of-the-art performance.

Contribution

The paper proposes a new training approach that omits initial SFT, using RL to activate reasoning capabilities, then applying SFT to address sampling inefficiencies and capability gaps.

Findings

Achieves state-of-the-art results on the OpenCompass Multimodal Reasoning Leaderboard.

Demonstrates effectiveness of RL incentivization without initial SFT.

Shows improved reasoning performance in 7B and 72B parameter models.

Abstract

Recent advancements in large language models (LLMs) have witnessed a surge in the development of advanced reasoning paradigms, which are now being integrated into multimodal large language models (MLLMs). However, existing approaches often fall short: methods solely employing reinforcement learning (RL) can struggle with sample inefficiency and activating entirely absent reasoning capabilities, while conventional pipelines that initiate with a cold-start supervised fine-tuning (SFT) phase before RL may restrict the model's exploratory capacity and face suboptimal convergence. In this work, we introduce \textbf{Metis-RISE} (\textbf{R}L \textbf{I}ncentivizes and \textbf{S}FT \textbf{E}nhances) for multimodal reasoning model learning. Unlike conventional approaches, Metis-RISE distinctively omits an initial SFT stage, beginning instead with an RL phase (e.g., using a Group Relative Policy Optimization variant) to incentivize and activate the model's latent reasoning capacity. Subsequently, the targeted SFT stage addresses two key challenges identified during RL: (1) \textit{inefficient trajectory sampling} for tasks where the model possesses but inconsistently applies correct reasoning, which we tackle using self-distilled reasoning trajectories from the RL model itself; and (2) \textit{fundamental capability absence}, which we address by injecting expert-augmented knowledge for prompts where the model entirely fails. This strategic application of RL for incentivization followed by SFT for enhancement forms the core of Metis-RISE, leading to two versions of our MLLMs (7B and 72B parameters). Evaluations on the OpenCompass Multimodal Reasoning Leaderboard demonstrate that both models achieve state-of-the-art performance among similar-sized models, with the 72B version ranking fourth overall. Please refer to our project page for open-source information.