CoLVR: Enhancing Exploratory Latent Visual Reasoning via Contrastive Optimization

TL;DR

CoLVR introduces a contrastive training framework to enhance the exploratory visual reasoning of multimodal large language models, leading to improved performance on various benchmarks.

Contribution

It proposes a novel latent contrastive training method and a trajectory contrastive reward to foster diverse reasoning behaviors in latent visual reasoning.

Findings

Achieved 5.83% improvement on VSP

Achieved 8.00% improvement on Jigsaw

Outperformed existing models on out-of-domain benchmarks with 3.40% gain

Abstract

Due to the potential for exploratory reasoning of Latent Visual Reasoning, recent works tend to enable MLLMs (Multimodal Large Language Models) to perform visual reasoning by propagating continuous hidden states instead of decoding intermediate steps into discrete tokens. However, existing works typically rely on hard alignment objectives to force latent representations to match predefined visual features, thereby severely limiting the exploratory of latent reasoning process. To address this problem, we propose CoLVR (Contrastive Optimization for Latent Visual Reasoning). To obtain a more exploratory visual reasoning, CoLVR introduces a latent contrastive training framework. Firstly, CoLVR learns diverse and exploratory representations with a latent contrastive objective guided by angle-based perturbation, which expands the semantic latent space and avoids over-constrained embedding. Then, CoLVR employs a latent trajectory contrastive reward for RL (Reinforcement Learning) post-training to enable fine-grained optimization of latent visual reasoning process and thus fostering diverse reasoning behaviors. Experiments demonstrate that CoLVR significantly enhances the exploratory capability of latent representations, achieving average improvements of 5.83% on VSP and 8.00% on Jigsaw, while also outperforming existing latent models on out of domain benchmarks, with a 3.40% gain on MMStar. The data, codes, and models are released at https://github.com/Oscar-dzy/CoLVR.