Intelligent LiDAR Navigation: Leveraging External Information and Semantic Maps with LLM as Copilot

TL;DR

This paper introduces a novel approach to robot navigation by integrating Large Language Models as copilots, using semantic maps to incorporate external contextual information for more human-like navigation capabilities.

Contribution

It presents osmAG, a semantic topometric map representation, and demonstrates how LLMs can serve as copilots to enhance traditional robotic navigation systems with contextual understanding.

Findings

LLMs can effectively incorporate external information into navigation.

Semantic maps improve the contextual awareness of robotic systems.

The approach maintains robustness while adding human-like understanding.

Abstract

Traditional robot navigation systems primarily utilize occupancy grid maps and laser-based sensing technologies, as demonstrated by the popular move_base package in ROS. Unlike robots, humans navigate not only through spatial awareness and physical distances but also by integrating external information, such as elevator maintenance updates from public notification boards and experiential knowledge, like the need for special access through certain doors. With the development of Large Language Models (LLMs), which possesses text understanding and intelligence close to human performance, there is now an opportunity to infuse robot navigation systems with a level of understanding akin to human cognition. In this study, we propose using osmAG (Area Graph in OpensStreetMap textual format), an innovative semantic topometric hierarchical map representation, to bridge the gap between the capabilities of ROS move_base and the contextual understanding offered by LLMs. Our methodology employs LLMs as an actual copilot in robot navigation, enabling the integration of a broader range of informational inputs while maintaining the robustness of traditional robotic navigation systems. Our code, demo, map, experiment results can be accessed at https://github.com/xiexiexiaoxiexie/Intelligent-LiDAR-Navigation-LLM-as-Copilot.