Learning to gesticulate by observation using a deep generative approach

TL;DR

This paper presents a deep generative approach using GANs to enable humanoid robots to produce natural talking gestures based on human gesture recordings, resulting in diverse and expressive robot behaviors.

Contribution

It introduces a GAN-based system for gesture generation from human recordings, integrating kinematic translation for realistic robot motion.

Findings

Robot can produce a wide variety of natural gestures

Generated gestures are expressive and non-dreary

System demonstrates effective gesture imitation from recordings

Abstract

The goal of the system presented in this paper is to develop a natural talking gesture generation behavior for a humanoid robot, by feeding a Generative Adversarial Network (GAN) with human talking gestures recorded by a Kinect. A direct kinematic approach is used to translate from human poses to robot joint positions. The provided videos show that the robot is able to use a wide variety of gestures, offering a non-dreary, natural expression level.