Deep Sensorimotor Control by Imitating Predictive Models of Human Motion

TL;DR

This paper introduces a novel reinforcement learning method for robot sensorimotor control by imitating predictive models of human motion, enabling zero-shot transfer and outperforming existing methods across various robots and tasks.

Contribution

It presents a new approach that leverages human motion prediction models for robot learning, bypassing traditional retargeting and adversarial techniques.

Findings

Outperforms existing baselines significantly.

Works across multiple robots and tasks.

Can replace dense rewards and curricula in manipulation tasks.

Abstract

As the embodiment gap between a robot and a human narrows, new opportunities arise to leverage datasets of humans interacting with their surroundings for robot learning. We propose a novel technique for training sensorimotor policies with reinforcement learning by imitating predictive models of human motions. Our key insight is that the motion of keypoints on human-inspired robot end-effectors closely mirrors the motion of corresponding human body keypoints. This enables us to use a model trained to predict future motion on human data \emph{zero-shot} on robot data. We train sensorimotor policies to track the predictions of such a model, conditioned on a history of past robot states, while optimizing a relatively sparse task reward. This approach entirely bypasses gradient-based kinematic retargeting and adversarial losses, which limit existing methods from fully leveraging the scale and diversity of modern human-scene interaction datasets. Empirically, we find that our approach can work across robots and tasks, outperforming existing baselines by a large margin. In addition, we find that tracking a human motion model can substitute for carefully designed dense rewards and curricula in manipulation tasks. Code, data and qualitative results available at https://jirl-upenn.github.io/track_reward/.