ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space

TL;DR

ImitationNet is an unsupervised deep learning framework that creates a shared latent space for human and robot poses, enabling accurate motion retargeting without paired data and supporting diverse input modalities.

Contribution

The paper introduces a novel unsupervised method for human-to-robot motion retargeting using shared latent space and adaptive contrastive learning, without requiring paired datasets.

Findings

Outperforms existing methods in efficiency and precision

Supports diverse input modalities like text, videos, and key poses

Successfully implemented on a real robot with collision avoidance

Abstract

This paper introduces a novel deep-learning approach for human-to-robot motion retargeting, enabling robots to mimic human poses accurately. Contrary to prior deep-learning-based works, our method does not require paired human-to-robot data, which facilitates its translation to new robots. First, we construct a shared latent space between humans and robots via adaptive contrastive learning that takes advantage of a proposed cross-domain similarity metric between the human and robot poses. Additionally, we propose a consistency term to build a common latent space that captures the similarity of the poses with precision while allowing direct robot motion control from the latent space. For instance, we can generate in-between motion through simple linear interpolation between two projected human poses. We conduct a comprehensive evaluation of robot control from diverse modalities (i.e., texts, RGB videos, and key poses), which facilitates robot control for non-expert users. Our model outperforms existing works regarding human-to-robot retargeting in terms of efficiency and precision. Finally, we implemented our method in a real robot with self-collision avoidance through a whole-body controller to showcase the effectiveness of our approach. More information on our website https://evm7.github.io/UnsH2R/