emg2tendon: From sEMG Signals to Tendon Control in Musculoskeletal Hands

TL;DR

This paper introduces a large-scale dataset and a novel diffusion-based model to accurately map surface electromyography signals to tendon control in robotic hands, advancing dexterous manipulation capabilities.

Contribution

It provides the first extensive EMG-to-tendon control dataset and a new diffusion-based regression model, addressing previous limitations in tendon-driven robotic hand control.

Findings

The dataset includes 370 hours of data from 193 subjects.

The diffusion model outperforms baseline regression models.

The framework enables more accurate tendon control prediction.

Abstract

Tendon-driven robotic hands offer unparalleled dexterity for manipulation tasks, but learning control policies for such systems presents unique challenges. Unlike joint-actuated robotic hands, tendon-driven systems lack a direct one-to-one mapping between motion capture (mocap) data and tendon controls, making the learning process complex and expensive. Additionally, visual tracking methods for real-world applications are prone to occlusions and inaccuracies, further complicating joint tracking. Wrist-wearable surface electromyography (sEMG) sensors present an inexpensive, robust alternative to capture hand motion. However, mapping sEMG signals to tendon control remains a significant challenge despite the availability of EMG-to-pose data sets and regression-based models in the existing literature. We introduce the first large-scale EMG-to-Tendon Control dataset for robotic hands, extending the emg2pose dataset, which includes recordings from 193 subjects, spanning 370 hours and 29 stages with diverse gestures. This dataset incorporates tendon control signals derived using the MyoSuite MyoHand model, addressing limitations such as invalid poses in prior methods. We provide three baseline regression models to demonstrate emg2tendon utility and propose a novel diffusion-based regression model for predicting tendon control from sEMG recordings. This dataset and modeling framework marks a significant step forward for tendon-driven dexterous robotic manipulation, laying the groundwork for scalable and accurate tendon control in robotic hands. https://emg2tendon.github.io/