Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Arms

TL;DR

This paper introduces a novel distance measure to address the correspondence problem in imitation learning between dissimilar robotic arms, enabling effective static and dynamic movement imitation through reinforcement learning.

Contribution

It proposes a new embodiment distance measure used as a loss function and feedback signal, improving imitation learning between different robotic morphologies.

Findings

The measure effectively captures similarity between dissimilar embodiments.

The approach enables successful static pose imitation.

It facilitates dynamic movement imitation via reinforcement learning.

Abstract

The development of autonomous robotic systems that can learn from human demonstrations to imitate a desired behavior - rather than being manually programmed - has huge technological potential. One major challenge in imitation learning is the correspondence problem: how to establish corresponding states and actions between expert and learner, when the embodiments of the agents are different (morphology, dynamics, degrees of freedom, etc.). Many existing approaches in imitation learning circumvent the correspondence problem, for example, kinesthetic teaching or teleoperation, which are performed on the robot. In this work we explicitly address the correspondence problem by introducing a distance measure between dissimilar embodiments. This measure is then used as a loss function for static pose imitation and as a feedback signal within a model-free deep reinforcement learning framework for dynamic movement imitation between two anthropomorphic robotic arms in simulation. We find that the measure is well suited for describing the similarity between embodiments and for learning imitation policies by distance minimization.