DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization

TL;DR

DexEMG introduces a lightweight, cost-effective EMG-based teleoperation system that accurately predicts hand movements and generalizes well across tasks without extensive calibration, enabling dexterous robotic control in unstructured environments.

Contribution

The paper presents EMG2Pose, a neural network for real-time hand pose prediction from EMG signals, and a robust retargeting algorithm, advancing EMG-based teleoperation with high accuracy and generalization.

Findings

Achieves high precision in diverse teleoperation tasks

Demonstrates strong generalization across objects and environments

Operates without extensive individual-specific recalibration

Abstract

High-fidelity teleoperation of dexterous robotic hands is essential for bringing robots into unstructured domestic environments. However, existing teleoperation systems often face a trade-off between performance and portability: vision-based capture systems are constrained by costs and line-of-sight requirements, while mechanical exoskeletons are bulky and physically restrictive. In this paper, we present DexEMG, a lightweight and cost-effective teleoperation system leveraging surface electromyography (sEMG) to bridge the gap between human intent and robotic execution. We first collect a synchronized dataset of sEMG signals and hand poses via a MoCap glove to train EMG2Pose, a neural network capable of continuously predicting hand kinematics directly from muscle activity. To ensure seamless control, we develop a robust hand retargeting algorithm that maps the predicted poses onto a multi-fingered dexterous hand in real-time. Experimental results demonstrate that DexEMG achieves high precision in diverse teleoperation tasks. Notably, our system exhibits strong generalization capabilities across novel objects and complex environments without the need for intensive individual-specific recalibration. This work offers a scalable and intuitive interface for both general-purpose robotic manipulation and assistive technologies.