Reflex-Based Open-Vocabulary Navigation without Prior Knowledge Using Omnidirectional Camera and Multiple Vision-Language Models

TL;DR

This paper presents a simple, map-free robot navigation method using an omnidirectional camera and pre-trained vision-language models, enabling open-vocabulary navigation without prior knowledge or learning.

Contribution

The study introduces a novel approach combining omnidirectional vision and multiple pre-trained vision-language models for map-free, prior-knowledge-free robot navigation.

Findings

Navigation achieved without prior maps or learning.

Omnidirectional camera simplifies environment perception.

Method demonstrates effective open-vocabulary navigation.

Abstract

Various robot navigation methods have been developed, but they are mainly based on Simultaneous Localization and Mapping (SLAM), reinforcement learning, etc., which require prior map construction or learning. In this study, we consider the simplest method that does not require any map construction or learning, and execute open-vocabulary navigation of robots without any prior knowledge to do this. We applied an omnidirectional camera and pre-trained vision-language models to the robot. The omnidirectional camera provides a uniform view of the surroundings, thus eliminating the need for complicated exploratory behaviors including trajectory generation. By applying multiple pre-trained vision-language models to this omnidirectional image and incorporating reflective behaviors, we show that navigation becomes simple and does not require any prior setup. Interesting properties and limitations of our method are discussed based on experiments with the mobile robot Fetch.