A Modular Approach to the Embodiment of Hand Motions from Human Demonstrations

TL;DR

This paper introduces a modular framework that automatically maps human hand motions to robotic hands, enabling effective manipulation of objects with improved success rates demonstrated in simulation and real-world tests.

Contribution

It presents a novel automatic embodiment mapping method that transfers human demonstrations to robotic systems for complex object manipulation tasks.

Findings

Successful transfer of human motions to robotic hands

Comparable success rates in simulation and real-world experiments

Effective manipulation of deformable and fragile objects

Abstract

Manipulating objects with robotic hands is a complicated task. Not only the fingers of the hand, but also the pose of the robot's end effector need to be coordinated. Using human demonstrations of movements is an intuitive and data-efficient way of guiding the robot's behavior. We propose a modular framework with an automatic embodiment mapping to transfer recorded human hand motions to robotic systems. In this work, we use motion capture to record human motion. We evaluate our approach on eight challenging tasks, in which a robotic hand needs to grasp and manipulate either deformable or small and fragile objects. We test a subset of trajectories in simulation and on a real robot and the overall success rates are aligned.