Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning

TL;DR

This paper investigates how visual information is integrated during reasoning in multimodal large language models, revealing that a small subset of tokens with strong visual-textual coupling serve as anchors, and introduces a new reinforcement learning framework to enhance this grounding.

Contribution

It uncovers the role of high-connectivity tokens as anchors in multimodal reasoning and proposes Anchor-Token Reinforcement Learning (AT-RL) to improve visual grounding with minimal overhead.

Findings

High-connectivity tokens are key anchors in reasoning.

AT-RL improves model performance on multiple tasks.

Focusing on low-connectivity tokens degrades reasoning quality.

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet how visual evidence is integrated during reasoning remains poorly understood. We explore multimodal RLVR through the lens of cross-modal attention connectivity and find that only a small fraction of tokens (approximately 15%) exhibit strong visual-textual coupling. These high-connectivity tokens act as anchors that ground reasoning in the image, while the majority follow linguistic patterns. During RLVR training, credit assignment naturally concentrates on these anchors, sharpening their visual grounding over time. Building on this insight, we propose Anchor-Token Reinforcement Learning (AT-RL), a lightweight framework that selectively reinforces high-connectivity tokens via graph-based clustering of attention topology. Evaluated across the series (3B-32B), AT-RL introduces only 1.2% overhead yet enables the 32B model to surpass the 72B-Instruct baseline on MathVista (80.2), with consistent gains observed across STEM, video and general tasks. Conversely, training solely on low-connectivity tokens causes severe degradation, confirming that effective multimodal RL hinges on precise credit assignment to visual anchors. Our work reveals that reasoning quality is governed not by token quantity but by the fidelity of cross-modal anchoring.