Bimanual High-Density EMG Control for In-Home Mobile Manipulation by a User with Quadriplegia

TL;DR

This paper presents a novel bimanual high-density EMG system enabling a user with quadriplegia to control a mobile manipulator at home, integrating shared autonomy for robust daily task execution.

Contribution

The work introduces the first wearable HDEMG system for home-based robot control by quadriplegic users, combining gesture recognition with shared autonomy in a real-world setting.

Findings

Successful 12-day in-home user study demonstrating practical robot control.

Effective gesture-based control of mobile manipulators for daily tasks.

Enhanced robustness through integrated vision, language, and motion planning.

Abstract

Mobile manipulators in the home can enable people with cervical spinal cord injury (cSCI) to perform daily physical household tasks that they could not otherwise do themselves. However, paralysis in these users often limits access to traditional robot control interfaces such as joysticks or keyboards. In this work, we introduce and deploy the first system that enables a user with quadriplegia to control a mobile manipulator in their own home using bimanual high-density electromyography (HDEMG). We develop a pair of custom, fabric-integrated HDEMG forearm sleeves, worn on both arms, that capture residual neuromotor activity from clinically paralyzed degrees of freedom and support real-time gesture-based robot control. Second, by integrating vision, language, and motion planning modules, we introduce a shared autonomy framework that supports robust and user-driven teleoperation, with particular benefits for navigation-intensive tasks in home environments. Finally, to demonstrate the system in the wild, we present a twelve-day in-home user study evaluating real-time use of the wearable EMG interface for daily robot control. Together, these system components enable effective robot control for performing activities of daily living and other household tasks in a real home environment.