DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes

TL;DR

DualMap is an innovative online semantic mapping system that allows robots to understand and navigate changing environments using natural language, combining efficient scene segmentation with dual-map representations for improved performance.

Contribution

The paper introduces DualMap, a novel hybrid segmentation frontend and dual-map system that enhances online semantic mapping and navigation in dynamic scenes, reducing computational costs.

Findings

State-of-the-art 3D open-vocabulary segmentation

Efficient online scene mapping in dynamic environments

Successful real-world and simulation navigation experiments

Abstract

We introduce DualMap, an online open-vocabulary mapping system that enables robots to understand and navigate dynamically changing environments through natural language queries. Designed for efficient semantic mapping and adaptability to changing environments, DualMap meets the essential requirements for real-world robot navigation applications. Our proposed hybrid segmentation frontend and object-level status check eliminate the costly 3D object merging required by prior methods, enabling efficient online scene mapping. The dual-map representation combines a global abstract map for high-level candidate selection with a local concrete map for precise goal-reaching, effectively managing and updating dynamic changes in the environment. Through extensive experiments in both simulation and real-world scenarios, we demonstrate state-of-the-art performance in 3D open-vocabulary segmentation, efficient scene mapping, and online language-guided navigation. Project page: https://eku127.github.io/DualMap/