VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation

TL;DR

This paper introduces VISOR, a compact vision-language-action agent that performs explicit, image-grounded reasoning for language-driven object navigation, improving explainability and generalization over existing methods.

Contribution

The paper presents a novel 3B-parameter VLA agent that replaces multi-model pipelines with explicit reasoning stages for better interpretability and efficiency in object navigation.

Findings

Enhanced explainability through explicit reasoning stages

Improved generalization to unseen objects and environments

More efficient navigation compared to prior methods

Abstract

Language-driven object navigation requires agents to interpret natural language descriptions of target objects, which combine intrinsic and extrinsic attributes for instance recognition and commonsense navigation. Existing methods either (i) use end-to-end trained models with vision-language embeddings, which struggle to generalize beyond training data and lack action-level explainability, or (ii) rely on modular zero-shot pipelines with large language models (LLMs) and open-set object detectors, which suffer from error propagation, high computational cost, and difficulty integrating their reasoning back into the navigation policy. To this end, we propose a compact 3B-parameter Vision-Language-Action (VLA) agent that performs human-like embodied reasoning for both object recognition and action selection, removing the need for stitched multi-model pipelines. Instead of raw embedding matching, our agent employs explicit image-grounded reasoning to directly answer "Is this the target object?" and "Why should I take this action?" The reasoning process unfolds in three stages: "think", "think summary", and "action", yielding improved explainability, stronger generalization, and more efficient navigation. Code and dataset available upon acceptance.