Self-Supervised Motion Retargeting with Safety Guarantee

TL;DR

This paper introduces S3LE, a self-supervised motion retargeting method that generates natural, collision-free humanoid robot motions from motion capture data or videos, with minimal data collection.

Contribution

It proposes a novel self-supervised learning framework with paired data generation and projection-invariant mapping, ensuring safe and expressive robot motions.

Findings

Successfully retargets motions from CMU database and YouTube videos.

Guarantees collision-free and limit-satisfying robot poses.

Reduces data collection effort for motion retargeting.

Abstract

In this paper, we present self-supervised shared latent embedding (S3LE), a data-driven motion retargeting method that enables the generation of natural motions in humanoid robots from motion capture data or RGB videos. While it requires paired data consisting of human poses and their corresponding robot configurations, it significantly alleviates the necessity of time-consuming data-collection via novel paired data generating processes. Our self-supervised learning procedure consists of two steps: automatically generating paired data to bootstrap the motion retargeting, and learning a projection-invariant mapping to handle the different expressivity of humans and humanoid robots. Furthermore, our method guarantees that the generated robot pose is collision-free and satisfies position limits by utilizing nonparametric regression in the shared latent space. We demonstrate that our method can generate expressive robotic motions from both the CMU motion capture database and YouTube videos.