In-Mouth Robotic Bite Transfer with Visual and Haptic Sensing

TL;DR

This paper presents a semi-autonomous robotic system for safe, comfortable, and effective in-mouth food transfer using visual and haptic sensing, enhancing independence for individuals with mobility issues.

Contribution

The work introduces a force-reactive control system and a novel dexterous wrist-like end effector for improved in-mouth food transfer in assistive robotics.

Findings

Users prefer the proposed method over baselines

System successfully detects bites and adapts to user motions

Participants find the system safe and comfortable

Abstract

Assistance during eating is essential for those with severe mobility issues or eating risks. However, dependence on traditional human caregivers is linked to malnutrition, weight loss, and low self-esteem. For those who require eating assistance, a semi-autonomous robotic platform can provide independence and a healthier lifestyle. We demonstrate an essential capability of this platform: safe, comfortable, and effective transfer of a bite-sized food item from a utensil directly to the inside of a person's mouth. Our system uses a force-reactive controller to safely accommodate the user's motions throughout the transfer, allowing full reactivity until bite detection then reducing reactivity in the direction of exit. Additionally, we introduce a novel dexterous wrist-like end effector capable of small, unimposing movements to reduce user discomfort. We conduct a user study with 11 participants covering 8 diverse food categories to evaluate our system end-to-end, and we find that users strongly prefer our method to a wide range of baselines. Appendices and videos are available at our website: https://tinyurl.com/btICRA.