Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning

TL;DR

This paper introduces RIS, a spatial-semantic grounded framework for latent visual reasoning in multimodal large language models, addressing information bottlenecks and manifold compatibility issues.

Contribution

RIS develops a grounded latent reasoning method that aligns with pretrained models using spatial and semantic evidence, improving interpretability and reasoning fidelity.

Findings

RIS achieves consistent improvements on multiple benchmarks.

RIS learns diverse and interpretable latent trajectories.

RIS enhances internal visual reasoning in MLLMs.

Abstract

Multimodal Large Language Models (MLLMs) have made remarkable progress on vision-language reasoning, yet most methods still compress visual evidence into discrete textual thoughts, creating an information bottleneck for fine-grained perception. Recent latent visual reasoning methods attempt to reason in continuous hidden states, but we find that they suffer from insufficient manifold compatibility: latent trajectories drift away from pretrained reasoning circuits, collapse into instance-agnostic patterns, and are often bypassed during answer generation. To address these issues, we propose RIS (Retrieve, Integrate, and Synthesize), a spatial-semantic grounded framework that develops latent reasoning as a compatible extension of pretrained MLLM computation. We first construct a step-wise grounded reasoning dataset with bounding boxes and region-specific semantic descriptions. Built on this supervision, RIS anchors latent tokens to both spatial and semantic evidence, enforces their causal role through a progressive attention bottleneck, and introduces short language transition tokens to bridge synthesized latent states back to vocabulary-aligned decoding. Experiments on V*, HRBench4K, HRBench8K, MMVP, and BLINK show consistent improvements over closed/open-source and latent reasoning baselines. Further analyses demonstrate that RIS learns diverse, interpretable, and progressively integrated latent trajectories, offering a practical path toward faithful internal visual reasoning in MLLMs.