HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration

TL;DR

This paper introduces HERD, a continuous evolution framework that transfers manipulation skills from humans to robots using micro-evolutionary reinforcement learning and path optimization, enabling effective policy transfer.

Contribution

HERD presents a novel continuous evolution approach combining reinforcement learning and path searching to transfer human manipulation skills to robots.

Findings

Effective transfer of human manipulation policies to robots

Continuous evolution improves policy adaptation

Framework handles high-dimensional robot parameters

Abstract

The ability to learn from human demonstration endows robots with the ability to automate various tasks. However, directly learning from human demonstration is challenging since the structure of the human hand can be very different from the desired robot gripper. In this work, we show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning, where a five-finger human dexterous hand robot gradually evolves into a commercial robot, while repeated interacting in a physics simulator to continuously update the policy that is first learned from human demonstration. To deal with the high dimensions of robot parameters, we propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy. Through experiments on human object manipulation datasets, we show that our framework can efficiently transfer the expert human agent policy trained from human demonstrations in diverse modalities to target commercial robots.