VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observations

TL;DR

This paper introduces VIEW2SPACE, a new benchmark for multi-view visual reasoning using simulated 3D scenes, revealing current models' limitations and proposing methods to improve reasoning across sparse views.

Contribution

The paper presents a scalable simulation-based benchmark for multi-view reasoning and evaluates state-of-the-art models, highlighting the challenges and proposing grounded chain-of-thought methods for improvement.

Findings

Multi-view reasoning models perform only marginally better than random.

Grounded Chain-of-Thought improves performance on moderate difficulty questions.

Scaling models benefits geometric perception but not deep reasoning across sparse views.

Abstract

Multi-view visual reasoning is essential for intelligent systems that must understand complex environments from sparse and discrete viewpoints, yet existing research has largely focused on single-image or temporally dense video settings. In real-world scenarios, reasoning across views requires integrating partial observations without explicit guidance, while collecting large-scale multi-view data with accurate geometric and semantic annotations remains challenging. To address this gap, we leverage physically grounded simulation to construct diverse, high-fidelity 3D scenes with precise per-view metadata, enabling scalable data generation that remains transferable to real-world settings. Based on this engine, we introduce VIEW2SPACE, a multi-dimensional benchmark for sparse multi-view reasoning, together with a scalable, disjoint training split supporting millions of grounded question-answer pairs. Using this benchmark, a comprehensive evaluation of state-of-the-art vision-language and spatial models reveals that multi-view reasoning remains largely unsolved, with most models performing only marginally above random guessing. We further investigate whether training can bridge this gap. Our proposed Grounded Chain-of-Thought with Visual Evidence substantially improves performance under moderate difficulty, and generalizes to real-world data, outperforming existing approaches in cross-dataset evaluation. We further conduct difficulty-aware scaling analyses across model size, data scale, reasoning depth, and visibility constraints, indicating that while geometric perception can benefit from scaling under sufficient visibility, deep compositional reasoning across sparse views remains a fundamental challenge.