ReMemNav: A Rethinking and Memory-Augmented Framework for Zero-Shot Object Navigation

TL;DR

ReMemNav is a hierarchical framework that enhances zero-shot object navigation by integrating panoramic semantic priors, episodic memory, and vision-language models to improve success rates and exploration efficiency.

Contribution

It introduces a novel memory-augmented hierarchical framework with a dual-modal rethinking mechanism and a new spatial reasoning model for zero-shot object navigation.

Findings

ReMemNav outperforms existing zero-shot baselines in success rate and exploration efficiency.

Achieves 1.7%-18.2% improvements in success rate across datasets.

Demonstrates significant improvements in success rate and SPL metrics.

Abstract

Zero-shot object navigation requires agents to locate unseen target objects in unfamiliar environments without prior maps or task-specific training which remains a significant challenge. Although recent advancements in vision-language models(VLMs) provide promising commonsense reasoning capabilities for this task, these models still suffer from spatial hallucinations, local exploration deadlocks, and a disconnect between high-level semantic intent and low-level control. In this regard, we propose a novel hierarchical navigation framework named ReMemNav, which seamlessly integrates panoramic semantic priors and episodic memory with VLMs. We introduce the Recognize Anything Model to anchor the spatial reasoning process of the VLM. We also design an adaptive dual-modal rethinking mechanism based on an episodic semantic buffer queue. The proposed mechanism actively verifies target visibility and corrects decisions using historical memory to prevent deadlocks. For low-level action execution, ReMemNav extracts a sequence of feasible actions using depth masks, allowing the VLM to select the optimal action for mapping into actual spatial movement. Extensive evaluations on HM3D and MP3D demonstrate that ReMemNav outperforms existing training-free zero-shot baselines in both success rate and exploration efficiency. Specifically, we achieve significant absolute performance improvements, with SR and SPL increasing by 1.7% and 7.0% on HM3D v0.1, 18.2% and 11.1% on HM3D v0.2, and 8.7% and 7.9% on MP3D.