LEGATO: Cross-Embodiment Imitation Using a Grasping Tool

TL;DR

LEGATO is a framework that enables the transfer of visuomotor skills across different robot embodiments by using a unified gripper interface and motion retargeting, facilitating scalable and reusable imitation learning.

Contribution

We introduce a novel cross-embodiment imitation learning framework using a unified gripper and motion retargeting, allowing skill transfer across diverse robot morphologies.

Findings

Effective transfer of visuomotor skills demonstrated in simulation and real robots.

Unified gripper design simplifies cross-robot task definition.

Retargeted motions achieve high accuracy across varied embodiments.

Abstract

Cross-embodiment imitation learning enables policies trained on specific embodiments to transfer across different robots, unlocking the potential for large-scale imitation learning that is both cost-effective and highly reusable. This paper presents LEGATO, a cross-embodiment imitation learning framework for visuomotor skill transfer across varied kinematic morphologies. We introduce a handheld gripper that unifies action and observation spaces, allowing tasks to be defined consistently across robots. We train visuomotor policies on task demonstrations using this gripper through imitation learning, applying transformation to a motion-invariant space for computing the training loss. Gripper motions generated by the policies are retargeted into high-degree-of-freedom whole-body motions using inverse kinematics for deployment across diverse embodiments. Our evaluations in simulation and real-robot experiments highlight the framework's effectiveness in learning and transferring visuomotor skills across various robots. More information can be found on the project page: https://ut-hcrl.github.io/LEGATO.