Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generation

TL;DR

Video4Spatial demonstrates that video diffusion models conditioned on scene context can perform complex spatial reasoning tasks like navigation and object grounding using only visual data, advancing visuospatial intelligence.

Contribution

The paper introduces Video4Spatial, a novel framework showing that video generative models can exhibit visuospatial reasoning without auxiliary data.

Findings

Successfully performs scene navigation and object grounding tasks.

Maintains spatial consistency and generalizes to new environments.

Demonstrates strong spatial understanding from video context.

Abstract

We investigate whether video generative models can exhibit visuospatial intelligence, a capability central to human cognition, using only visual data. To this end, we present Video4Spatial, a framework showing that video diffusion models conditioned solely on video-based scene context can perform complex spatial tasks. We validate on two tasks: scene navigation - following camera-pose instructions while remaining consistent with 3D geometry of the scene, and object grounding - which requires semantic localization, instruction following, and planning. Both tasks use video-only inputs, without auxiliary modalities such as depth or poses. With simple yet effective design choices in the framework and data curation, Video4Spatial demonstrates strong spatial understanding from video context: it plans navigation and grounds target objects end-to-end, follows camera-pose instructions while maintaining spatial consistency, and generalizes to long contexts and out-of-domain environments. Taken together, these results advance video generative models toward general visuospatial reasoning.