Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch

TL;DR

This paper introduces SILEM, a novel imitation learning method that compensates for embodiment mismatch by learning affine transforms on skeletal features, enabling robots to learn from diverse demonstrations including videos.

Contribution

The paper proposes SILEM, a new technique that addresses embodiment mismatch in imitation learning by applying learned affine transformations to skeletal features.

Findings

SILEM improves imitation performance in toy and realistic domains.

It enables humanoid robots to learn walking from human demonstrations.

SILEM effectively compensates for body differences in imitation tasks.

Abstract

Learning from demonstrations in the wild (e.g. YouTube videos) is a tantalizing goal in imitation learning. However, for this goal to be achieved, imitation learning algorithms must deal with the fact that the demonstrators and learners may have bodies that differ from one another. This condition -- "embodiment mismatch" -- is ignored by many recent imitation learning algorithms. Our proposed imitation learning technique, SILEM (\textbf{S}keletal feature compensation for \textbf{I}mitation \textbf{L}earning with \textbf{E}mbodiment \textbf{M}ismatch), addresses a particular type of embodiment mismatch by introducing a learned affine transform to compensate for differences in the skeletal features obtained from the learner and expert. We create toy domains based on PyBullet's HalfCheetah and Ant to assess SILEM's benefits for this type of embodiment mismatch. We also provide qualitative and quantitative results on more realistic problems -- teaching simulated humanoid agents, including Atlas from Boston Dynamics, to walk by observing human demonstrations.