From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoning

TL;DR

This paper introduces Visual Rationale Learning (ViRL), a new paradigm for vision-language reasoning that grounds models in visual rationales, improving transparency and accuracy by treating visual actions as core reasoning primitives.

Contribution

The paper proposes ViRL, an end-to-end reinforcement learning framework that incorporates process supervision, objective alignment, and fine-grained credit assignment for better visual reasoning.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Enhances model transparency and trustworthiness.

Effectively grounds reasoning in visual evidence.

Abstract

Recent advances in vision-language reasoning underscore the importance of thinking with images, where models actively ground their reasoning in visual evidence. Yet, prevailing frameworks treat visual actions as optional tools, boosting metrics but leaving reasoning ungrounded and crops ineffective. This gap gives rise to the illusion of thinking with images: models seem visually grounded but rely on context-agnostic actions that neither refine perception nor guide reasoning toward correct answers. We address this problem by reframing visual actions as core reasoning primitives rather than optional tools, which we term visual rationalization, the visual analogue of textual Chain-of-Thought. Building on this insight, we propose Visual Rationale Learning (ViRL), an end-to-end paradigm that grounds training in the visual rationale itself. ViRL integrates (1) Process Supervision with ground-truth rationales, (2) Objective Alignment via step-level reward shaping, and (3) Fine-Grained Credit Assignment to distinguish correct, redundant, and erroneous actions. By ensuring each action contributes meaningfully to the reasoning chain, ViRL enables models to "get the right answer for the right visual reason". Trained purely with end-to-end RL, ViRL achieves state-of-the-art results across benchmarks spanning perception, hallucination, and reasoning. This work establishes visual rationalization as a task-agnostic, process-grounded paradigm for building transparent, verifiable, and trustworthy vision-language models.