Uncertainty-Informed Active Perception for Open Vocabulary Object Goal Navigation

TL;DR

This paper introduces a probabilistic perception framework that quantifies semantic uncertainty in vision-language models to improve open vocabulary object goal navigation in indoor environments, leading to more efficient exploration.

Contribution

It presents a novel semantic uncertainty model and integrates it into a probabilistic map and exploration planner for better robot navigation without extensive prompt engineering.

Findings

Achieves success rates comparable to state-of-the-art methods.

Effectively quantifies semantic uncertainty in vision-language perception.

Enhances exploration efficiency in indoor object navigation.

Abstract

Mobile robots exploring indoor environments increasingly rely on vision-language models to perceive high-level semantic cues in camera images, such as object categories. Such models offer the potential to substantially advance robot behaviour for tasks such as object-goal navigation (ObjectNav), where the robot must locate objects specified in natural language by exploring the environment. Current ObjectNav methods heavily depend on prompt engineering for perception and do not address the semantic uncertainty induced by variations in prompt phrasing. Ignoring semantic uncertainty can lead to suboptimal exploration, which in turn limits performance. Hence, we propose a semantic uncertainty-informed active perception pipeline for ObjectNav in indoor environments. We introduce a novel probabilistic sensor model for quantifying semantic uncertainty in vision-language models and incorporate it into a probabilistic geometric-semantic map to enhance spatial understanding. Based on this map, we develop a frontier exploration planner with an uncertainty-informed multi-armed bandit objective to guide efficient object search. Experimental results demonstrate that our method achieves ObjectNav success rates comparable to those of state-of-the-art approaches, without requiring extensive prompt engineering.