Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments

TL;DR

Guide-LLM is an innovative embodied LLM agent that uses a text-based topological map and commonsense reasoning to provide personalized, efficient indoor navigation guidance for visually impaired individuals, surpassing traditional aids.

Contribution

This paper introduces a novel embodied LLM-based navigation system with a text-based topological map and hazard detection, advancing assistive technology for PVI in indoor environments.

Findings

Effective global path planning using simplified topological maps

Enhanced hazard detection through LLM commonsense reasoning

Personalized navigation assistance demonstrating adaptability

Abstract

Navigation presents a significant challenge for persons with visual impairments (PVI). While traditional aids such as white canes and guide dogs are invaluable, they fall short in delivering detailed spatial information and precise guidance to desired locations. Recent developments in large language models (LLMs) and vision-language models (VLMs) offer new avenues for enhancing assistive navigation. In this paper, we introduce Guide-LLM, an embodied LLM-based agent designed to assist PVI in navigating large indoor environments. Our approach features a novel text-based topological map that enables the LLM to plan global paths using a simplified environmental representation, focusing on straight paths and right-angle turns to facilitate navigation. Additionally, we utilize the LLM's commonsense reasoning for hazard detection and personalized path planning based on user preferences. Simulated experiments demonstrate the system's efficacy in guiding PVI, underscoring its potential as a significant advancement in assistive technology. The results highlight Guide-LLM's ability to offer efficient, adaptive, and personalized navigation assistance, pointing to promising advancements in this field.