Interleaved Latent Visual Reasoning with Selective Perceptual Modeling

TL;DR

ILVR introduces a novel framework for multimodal reasoning that combines dynamic latent visual state evolution with precise perceptual modeling, improving reasoning performance while reducing computational costs.

Contribution

ILVR unifies latent visual reasoning with interleaved textual generation, employing a self-supervised feature distillation strategy for adaptive visual cue generation.

Findings

ILVR outperforms existing methods on multimodal reasoning benchmarks.

The approach effectively balances perceptual detail and computational efficiency.

Extensive experiments validate the superiority of ILVR in dynamic visual reasoning.

Abstract

Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost of re-encoding pixel-dense images. A promising alternative, latent visual reasoning, circumvents this bottleneck yet faces limitations: methods either fail to capture intermediate state evolution due to single-step, non-interleaved structures, or sacrifice precise perceptual modeling by over-compressing features. We introduce Interleaved Latent Visual Reasoning (ILVR), a framework that unifies dynamic state evolution with precise perceptual modeling. ILVR interleaves textual generation with latent visual representations that act as specific, evolving cues for subsequent reasoning. Specifically, we employ a self-supervision strategy where a momentum teacher model selectively distills relevant features from ground-truth intermediate images into sparse supervision targets. This adaptive selection mechanism guides the model to autonomously generate context-aware visual signals. Extensive experiments on multimodal reasoning benchmarks demonstrate that ILVR outperforms existing approaches, effectively bridging the gap between fine-grained perception and sequential multimodal reasoning. The code is available at https://github.com/XD111ds/ILVR.