Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignment

TL;DR

This paper presents a method for transferring robot manipulation skills across different robot embodiments by aligning their state and action spaces in a shared latent space using generative adversarial training, enabling effective sim-to-real transfer.

Contribution

It introduces a novel latent space alignment approach with encoders and decoders trained via adversarial and cycle consistency losses for cross-embodiment policy transfer.

Findings

Successful sim-to-sim transfer between different robot models.

Effective sim-to-real transfer without reward tuning.

Robust policy transfer across varying robot morphologies.

Abstract

This paper focuses on transferring control policies between robot manipulators with different morphology. While reinforcement learning (RL) methods have shown successful results in robot manipulation tasks, transferring a trained policy from simulation to a real robot or deploying it on a robot with different states, actions, or kinematics is challenging. To achieve cross-embodiment policy transfer, our key insight is to project the state and action spaces of the source and target robots to a common latent space representation. We first introduce encoders and decoders to associate the states and actions of the source robot with a latent space. The encoders, decoders, and a latent space control policy are trained simultaneously using loss functions measuring task performance, latent dynamics consistency, and encoder-decoder ability to reconstruct the original states and actions. To transfer the learned control policy, we only need to train target encoders and decoders that align a new target domain to the latent space. We use generative adversarial training with cycle consistency and latent dynamics losses without access to the task reward or reward tuning in the target domain. We demonstrate sim-to-sim and sim-to-real manipulation policy transfer with source and target robots of different states, actions, and embodiments. The source code is available at \url{https://github.com/ExistentialRobotics/cross_embodiment_transfer}.