Towards an LLM-Based Speech Interface for Robot-Assisted Feeding

TL;DR

This paper presents an LLM-based speech interface for a robot-assisted feeding system, enabling natural communication for users with motor impairments, evaluated through a user study with older adults.

Contribution

It introduces a novel LLM-based speech interface framework for assistive robots, emphasizing human-centric design and practical evaluation.

Findings

Effective communication of high-level commands achieved

Positive user feedback from older adults

Demonstrated feasibility of LLMs in assistive robotics

Abstract

Physically assistive robots present an opportunity to significantly increase the well-being and independence of individuals with motor impairments or other forms of disability who are unable to complete activities of daily living (ADLs). Speech interfaces, especially ones that utilize Large Language Models (LLMs), can enable individuals to effectively and naturally communicate high-level commands and nuanced preferences to robots. In this work, we demonstrate an LLM-based speech interface for a commercially available assistive feeding robot. Our system is based on an iteratively designed framework, from the paper "VoicePilot: Harnessing LLMs as Speech Interfaces for Physically Assistive Robots," that incorporates human-centric elements for integrating LLMs as interfaces for robots. It has been evaluated through a user study with 11 older adults at an independent living facility. Videos are located on our project website: https://sites.google.com/andrew.cmu.edu/voicepilot/.