From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication
Yohei Hayamizu, David DeFazio, Hrudayangam Mehta, Zainab Altaweel, Jacqueline Choe, Chao Lin, Jake Juettner, Furui Xiao, Jeremy Blackburn, and Shiqi Zhang

TL;DR
This paper presents a novel dialog system for robotic guide dogs that uses large language models to verbalize navigation plans and scenes, enhancing communication and collaboration with visually impaired users.
Contribution
It introduces a new approach to grounding language in dynamic environments and improving spatial awareness in assistive robotic guide dogs using LLMs.
Findings
Verbalization strategies impact user understanding and trust.
The system improves navigation efficiency and accuracy.
Human studies validate the effectiveness of verbal communication.
Abstract
Assistive robotics is an important subarea of robotics that focuses on the well-being of people with disabilities. A robotic guide dog is an assistive quadruped robot that helps visually impaired people in obstacle avoidance and navigation. Enabling language capabilities for robotic guide dogs goes beyond naively adding an existing dialog system onto a mobile robot. The novel challenges include grounding language in the dynamically changing environment and improving spatial awareness for the human handler. To address those challenges, we develop a novel dialog system for robotic guide dogs that uses LLMs to verbalize both navigational plans and scenes. The goal is to enable verbal communication for collaborative decision-making within the handler-robot team. In experiments, we conducted a human study to evaluate different verbalization strategies and a simulation study to assess the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsSocial Robot Interaction and HRI · Human-Animal Interaction Studies · Gaze Tracking and Assistive Technology
