One RL to See Them All: Visual Triple Unified Reinforcement Learning

TL;DR

This paper introduces V-Triune, a unified multimodal reinforcement learning framework, and develops Orsta models that outperform specialized models across multiple vision-language tasks.

Contribution

It proposes a novel V-Triune methodology for unified multimodal RL and demonstrates its effectiveness with the Orsta models on diverse benchmarks.

Findings

Unified training matches or outperforms specialist models.

Orsta models improve over backbones on MEGA-Bench.

Unified RL enhances reasoning and perception in VLMs.

Abstract

Reinforcement learning (RL) is becoming an important direction for post-training vision-language models (VLMs), but public training methodologies for unified multimodal RL remain much less mature, especially for heterogeneous reasoning and perception-heavy tasks. We propose V-Triune, a Visual Triple Unified Reinforcement Learning methodology for unified multimodal RL. It organizes training around three coordinated abstractions: Sample-Level Reward Routing, Verifier-Level Outcome Verification, and Source-Level Diagnostics. Within this methodology, Dynamic IoU provides localization-specific reward shaping that avoids reward ambiguity under loose thresholds and reward sparsity under strict ones. Built on V-Triune, we develop Orsta (7B, 32B), a family of models jointly trained on eight reasoning and perception tasks. Under matched budgets, unified training matches or outperforms specialist mixtures. The final Orsta models improve over their backbones on MEGA-Bench, compare favorably with strong multi-task RL-VLM baselines, and transfer these gains to a broad set of downstream benchmarks. These results show that unified RL can improve both reasoning and perception within a single VLM RL pipeline.The V-Triune system, along with the Orsta models, is publicly available at https://github.com/MiniMax-AI/One-RL-to-See-Them-All.