A Modular Approach to the Embodiment of Hand Motions from Human Demonstrations
Alexander Fabisch, Manuela Uliano, Dennis Marschner, Melvin Laux,, Johannes Brust, Marco Controzzi

TL;DR
This paper introduces a modular framework that automatically maps human hand motions to robotic hands, enabling effective manipulation of objects with improved success rates demonstrated in simulation and real-world tests.
Contribution
It presents a novel automatic embodiment mapping method that transfers human demonstrations to robotic systems for complex object manipulation tasks.
Findings
Successful transfer of human motions to robotic hands
Comparable success rates in simulation and real-world experiments
Effective manipulation of deformable and fragile objects
Abstract
Manipulating objects with robotic hands is a complicated task. Not only the fingers of the hand, but also the pose of the robot's end effector need to be coordinated. Using human demonstrations of movements is an intuitive and data-efficient way of guiding the robot's behavior. We propose a modular framework with an automatic embodiment mapping to transfer recorded human hand motions to robotic systems. In this work, we use motion capture to record human motion. We evaluate our approach on eight challenging tasks, in which a robotic hand needs to grasp and manipulate either deformable or small and fragile objects. We test a subset of trajectories in simulation and on a real robot and the overall success rates are aligned.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Human Pose and Action Recognition · Human Motion and Animation
