Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models

TL;DR

This paper introduces GAP, a new paradigm for visual latent reasoning in multimodal large language models, addressing feature-space mismatch issues to improve stability and performance.

Contribution

GAP aligns visual latent reasoning at multiple levels, enhancing stability and performance in multimodal large language models without external tools.

Findings

Achieves best mean aggregate perception and reasoning on Qwen2.5-VL 7B.

Latent signals provide task-relevant visual information beyond token slots.

Addresses feature-space mismatch to improve latent feedback reliability.

Abstract

Visual latent reasoning lets a multimodal large language model (MLLM) create intermediate visual evidence as continuous tokens, avoiding external tools or image generators. However, existing methods usually follow an output-as-input latent paradigm and yield unstable gains. We identify evidence for a feature-space mismatch that can contribute to this instability: dominant visual-latent models build on pre-norm MLLMs and reuse decoder hidden states as predicted latent inputs, even though these states occupy a substantially different norm regime from the input embeddings the model was trained to consume~\citep{xie2025mhc,li2026siamesenorm,team2026attention}. This mismatch can make direct latent feedback unreliable. Motivated by this diagnosis, we propose \textbf{GAP}, a \textbf{G}ranular \textbf{A}lignment \textbf{P}aradigm for visual latent modeling. GAP aligns visual latent reasoning at three levels: feature-level alignment maps decoder outputs into input-compatible visual latents through a lightweight PCA-aligned latent head; context-level alignment grounds latent targets with inspectable auxiliary visual supervision; and capacity-guided alignment assigns latent supervision selectively to examples where the base MLLM struggles. On Qwen2.5-VL 7B, the resulting model achieves the best mean aggregate perception and reasoning performance among our supervised variants. Inference-time intervention probing further suggests that generated latents provide task-relevant visual signal beyond merely adding token slots.