Analyzing Key Objectives in Human-to-Robot Retargeting for Dexterous Manipulation

TL;DR

This paper conducts extensive real-world experiments to analyze and evaluate various retargeting objectives for transferring human hand motions to robot hands, aiming to improve dexterous manipulation.

Contribution

It introduces a comprehensive retargeting objective formulation and provides experimental insights into the significance of different factors affecting retargeting performance.

Findings

Certain retargeting factors significantly improve manipulation accuracy

Experimental ablation highlights key objectives for effective retargeting

Results guide future design of retargeting algorithms for dexterous robots

Abstract

Kinematic retargeting from human hands to robot hands is essential for transferring dexterity from humans to robots in manipulation teleoperation and imitation learning. However, due to mechanical differences between human and robot hands, completely reproducing human motions on robot hands is impossible. Existing works on retargeting incorporate various optimization objectives, focusing on different aspects of hand configuration. However, the lack of experimental comparative studies leaves the significance and effectiveness of these objectives unclear. This work aims to analyze these retargeting objectives for dexterous manipulation through extensive real-world comparative experiments. Specifically, we propose a comprehensive retargeting objective formulation that integrates intuitively crucial factors appearing in recent approaches. The significance of each factor is evaluated through experimental ablation studies on the full objective in kinematic posture retargeting and real-world teleoperated manipulation tasks. Experimental results and conclusions provide valuable insights for designing more accurate and effective retargeting algorithms for real-world dexterous manipulation.