Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning

TL;DR

PEARL enhances multimodal reasoning in vision-language models by anchoring reasoning to verified visual evidence, reducing hallucinations and improving accuracy through perception checks and reinforcement learning.

Contribution

This paper introduces PEARL, a novel perception-reasoning framework that explicitly incorporates visual evidence verification into reinforcement learning for multimodal models.

Findings

Achieves +9.7% improvement on MathVerse benchmark.

Reduces visual hallucinations and reasoning errors.

Effectively integrates with existing RL methods like GRPO and DAPO.

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capabilities of Large Language Models (LLMs) and is now being applied to Vision-Language Models (VLMs). However, vanilla RLVR for VLMs verifies only the final textual output, critically neglecting the foundational step of visual perception. This oversight leads to visual hallucinations and reward hacking, as reasoning built upon flawed perception is inherently unreliable. To address this, we propose PEARL (Perceptual-Evidence Anchored Reinforced Learning), a dual-branch, perception-reasoning synergistic that strengthens multimodal reasoning by explicitly anchoring it to verified visual evidence. For each reasoning-oriented QA instance, PEARL first derive a perception checklist -- a set of perception-oriented sub-questions with verifiable answers that probe the model's understanding of key visual evidence. During training, auxiliary rollouts on this checklist yield a perceptual reward that both directly reinforces the model's perception ability and acts as a fidelity gate for reasoning. If the model passes the perception check, its policy update is biased towards evidence-anchored reasoning. Otherwise, the process is halted to prevent reasoning from flawed premises. PEARL can be seamlessly integrated with popular RL methods like GRPO and DAPO. Comprehensive experiments show PEARL achieves substantial gains on multimodal reasoning benchmarks, e.g., a +9.7% improvement over the baseline and +6.6% over GRPO on MathVerse.