Spatial-Conditioned Reasoning in Long-Egocentric Videos

TL;DR

This paper investigates how explicit spatial signals and depth information enhance vision-language models' ability to understand long egocentric videos for navigation, highlighting trade-offs and improvements in safety-critical tasks.

Contribution

It introduces Sanpo-D, a detailed re-annotation of a dataset, and benchmarks the impact of spatial signals and depth fusion on spatial reasoning in egocentric videos.

Findings

Depth-aware representations improve pedestrian detection.

Spatial grounding enhances obstruction detection.

Trade-off exists between general accuracy and spatial specialization.

Abstract

Long-horizon egocentric video presents significant challenges for visual navigation due to viewpoint drift and the absence of persistent geometric context. Although recent vision-language models perform well on image and short-video reasoning, their spatial reasoning capability in long egocentric sequences remains limited. In this work, we study how explicit spatial signals influence VLM-based video understanding without modifying model architectures or inference procedures. We introduce Sanpo-D, a fine-grained re-annotation of the Google Sanpo dataset, and benchmark multiple VLMs on navigation-oriented spatial queries. To examine input-level inductive bias, we further fuse depth maps with RGB frames and evaluate their impact on spatial reasoning. Our results reveal a trade-off between general-purpose accuracy and spatial specialization, showing that depth-aware and spatially grounded representations can improve performance on safety-critical tasks such as pedestrian and obstruction detection.