DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning

TL;DR

DeepEyes is a reinforcement learning-based model that learns to integrate visual information into reasoning processes, improving multimodal understanding and reasoning without pre-collected data, mirroring human cognition.

Contribution

We introduce DeepEyes, a novel reinforcement learning framework enabling models to learn visual reasoning intrinsically without relying on external data or models.

Findings

Significant performance improvements on perception and reasoning benchmarks.

Enhanced grounding, reduced hallucination, and better mathematical reasoning.

Emergence of human-like visual reasoning patterns.

Abstract

Large Vision-Language Models excel at multimodal understanding but struggle to deeply integrate visual information into their predominantly text-based reasoning processes, a key challenge in mirroring human cognition. To address this, we introduce DeepEyes, a model that learns to "think with images", trained end-to-end with reinforcement learning without requiring pre-collected reasoning data for cold-start supervised fine-tuning (SFT). Notably, this ability emerges natively, leveraging the model's own grounding capability as an intrinsic function rather than relying on external specialized models or APIs. We enable this capability through active perception, where the model learns to strategically ground its reasoning in visual information, guided by a tailored data selection and reward strategy. DeepEyes achieves significant performance gains on general perception and reasoning benchmarks and also demonstrates improvement in grounding, hallucination, and mathematical reasoning tasks. Interestingly, we observe the distinct evolution of active perception from initial exploration to efficient and accurate exploitation, and diverse thinking patterns that closely mirror human visual reasoning processes. Code is available at https://github.com/Visual-Agent/DeepEyes.