Imitating by generating: deep generative models for imitation of interactive tasks

TL;DR

This paper presents a deep generative framework enabling robots to learn and imitate interactive social tasks by predicting and adapting to human partner behaviors through observational and kinesthetic learning.

Contribution

It introduces a novel deep learning approach combining motion embedding, prediction, and generation for human-robot interaction tasks, validated on multiple social activities.

Findings

Predictive and adaptive components improve imitation accuracy.

Low-level motion abstractions are crucial for successful imitation.

The framework effectively learns tasks like hand-shake and fist-bump.

Abstract

To coordinate actions with an interaction partner requires a constant exchange of sensorimotor signals. Humans acquire these skills in infancy and early childhood mostly by imitation learning and active engagement with a skilled partner. They require the ability to predict and adapt to one's partner during an interaction. In this work we want to explore these ideas in a human-robot interaction setting in which a robot is required to learn interactive tasks from a combination of observational and kinesthetic learning. To this end, we propose a deep learning framework consisting of a number of components for (1) human and robot motion embedding, (2) motion prediction of the human partner and (3) generation of robot joint trajectories matching the human motion. To test these ideas, we collect human-human interaction data and human-robot interaction data of four interactive tasks "hand-shake", "hand-wave", "parachute fist-bump" and "rocket fist-bump". We demonstrate experimentally the importance of predictive and adaptive components as well as low-level abstractions to successfully learn to imitate human behavior in interactive social tasks.