Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence

TL;DR

This paper introduces TwNV, a paradigm that enhances spatial reasoning in Large Multimodal Models by integrating generative novel-view synthesis, leading to consistent accuracy improvements across various tasks and architectures.

Contribution

The paper presents a systematic approach combining novel-view synthesis with reasoning to improve spatial understanding in LMMs, demonstrating significant accuracy gains.

Findings

Numerical camera-pose instructions outperform free-form language for view control.

Synthesized view quality directly impacts spatial reasoning accuracy.

Iterative multi-turn view refinement further enhances model performance.

Abstract

Current Large Multimodal Models (LMMs) struggle with spatial reasoning tasks requiring viewpoint-dependent understanding, largely because they are confined to a single, static observation. We propose Thinking with Novel Views (TwNV), a paradigm that integrates generative novel-view synthesis into the reasoning loop: a Reasoner LMM identifies spatial ambiguity, instructs a Painter to synthesize an alternative viewpoint, and re-examines the scene with the additional evidence. Through systematic experiments we address three research questions. (1) Instruction format: numerical camera-pose specifications yield more reliable view control than free-form language. (2) Generation fidelity: synthesized view quality is tightly coupled with downstream spatial accuracy. (3) Inference-time visual scaling: iterative multi-turn view refinement further improves performance, echoing recent scaling trends in language reasoning. Across four spatial subtask categories and four LMM architectures (both closed- and open-source), TwNV consistently improves accuracy by +1.3 to +3.9 pp, with the largest gains on viewpoint-sensitive subtasks. These results establish novel-view generation as a practical lever for advancing spatial intelligence of LMMs.