CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models

TL;DR

This paper introduces CaMo, a novel vision-language model that explicitly understands camera motion and spatial narratives, improving spatial reasoning and evaluation beyond traditional question answering benchmarks.

Contribution

The paper presents CaMo, a new model that incorporates camera motion grounding and a novel Spatial Narrative Score for better spatial cognition evaluation.

Findings

CaMo outperforms existing models on spatial narrative tasks.

State-of-the-art spatial VLMs degrade significantly under SNS evaluation.

Explicit spatial narrative externalization enhances 3D spatial understanding.

Abstract

Vision-Language Models (VLMs) achieve strong performance on spatial question answering benchmarks, yet it remains unclear whether such gains reflect genuine spatial intelligence. We show that existing spatial VLMs lack basic camera motion understanding, a key component of spatial cognition. We propose the Spatial Narrative Score (SNS), an evaluation framework that requires VLMs to generate explicit spatial narratives capturing both scene semantics and camera motion, followed by reasoning with a frozen proxy LLM. Under SNS, state-of-the-art spatial VLMs exhibit significant performance degradation despite high direct question answering accuracy. To address this gap, we introduce CaMo, a camera motion grounded VLM that achieves consistent performance across SNS evaluation and direct spatial question answering accuracy. Our results highlight the importance of explicit spatial narrative externalization for evaluating VLMs with transferable 3D spatial understanding. Our code, data, and model is available at https://github.com/hsiangwei0903/CaMo