SemNav: A Model-Based Planner for Zero-Shot Object Goal Navigation Using Vision-Foundation Models

TL;DR

This paper introduces SemNav, a zero-shot object goal navigation framework that combines vision foundation models with a model-based planner, enabling scalable and generalizable navigation without task-specific training.

Contribution

It presents a novel integration of vision foundation models with a model-based planner for zero-shot object navigation, achieving state-of-the-art results.

Findings

Achieves top performance on HM3D dataset for zero-shot navigation.

Outperforms existing methods in success weighted by path length.

Demonstrates effective scene understanding and object localization without training.

Abstract

Object goal navigation is a fundamental task in embodied AI, where an agent is instructed to locate a target object in an unexplored environment. Traditional learning-based methods rely heavily on large-scale annotated data or require extensive interaction with the environment in a reinforcement learning setting, often failing to generalize to novel environments and limiting scalability. To overcome these challenges, we explore a zero-shot setting where the agent operates without task-specific training, enabling more scalable and adaptable solution. Recent advances in Vision Foundation Models (VFMs) offer powerful capabilities for visual understanding and reasoning, making them ideal for agents to comprehend scenes, identify relevant regions, and infer the likely locations of objects. In this work, we present a zero-shot object goal navigation framework that integrates the perceptual strength of VFMs with a model-based planner that is capable of long-horizon decision making through frontier exploration. We evaluate our approach on the HM3D dataset using the Habitat simulator and demonstrate that our method achieves state-of-the-art performance in terms of success weighted by path length for zero-shot object goal navigation.