Nav-$R^2$ Dual-Relation Reasoning for Generalizable Open-Vocabulary Object-Goal Navigation

TL;DR

Nav-R^2 introduces a structured reasoning framework with memory components for open-vocabulary object-goal navigation, significantly improving unseen object localization success rates in unseen environments.

Contribution

The paper presents Nav-R^2, a novel reasoning-based approach with a new dataset and memory module, enhancing generalization and efficiency in open-vocabulary navigation tasks.

Findings

Achieves state-of-the-art unseen object localization performance.

Operates at 2Hz for real-time inference.

Effectively models relationships with structured reasoning and memory.

Abstract

Object-goal navigation in open-vocabulary settings requires agents to locate novel objects in unseen environments, yet existing approaches suffer from opaque decision-making processes and low success rate on locating unseen objects. To address these challenges, we propose Nav-$R^2$, a framework that explicitly models two critical types of relationships, target-environment modeling and environment-action planning, through structured Chain-of-Thought (CoT) reasoning coupled with a Similarity-Aware Memory. We construct a Nav$R^2$-CoT dataset that teaches the model to perceive the environment, focus on target-related objects in the surrounding context and finally make future action plans. Our SA-Mem preserves the most target-relevant and current observation-relevant features from both temporal and semantic perspectives by compressing video frames and fusing historical observations, while introducing no additional parameters. Compared to previous methods, Nav-R^2 achieves state-of-the-art performance in localizing unseen objects through a streamlined and efficient pipeline, avoiding overfitting to seen object categories while maintaining real-time inference at 2Hz. Resources will be made publicly available at \href{https://github.com/AMAP-EAI/Nav-R2}{github link}.