Nonverbal Social Behavior Generation for Social Robots Using End-to-End Learning

TL;DR

This paper introduces an end-to-end neural network approach using Seq2Seq and GANs to generate realistic nonverbal social behaviors in robots, enhancing human-robot interaction by making robot responses less predictable and more natural.

Contribution

It presents a novel neural network architecture that learns social behaviors from human interactions and employs GANs to improve long-term behavior validity, with new metrics for evaluation.

Findings

The method produces more natural social behaviors in robots.

Learning multiple behaviors improves interaction quality.

New metrics effectively evaluate behavior similarity.

Abstract

To provide effective and enjoyable human-robot interaction, it is important for social robots to exhibit nonverbal behaviors, such as a handshake or a hug. However, the traditional approach of reproducing pre-coded motions allows users to easily predict the reaction of the robot, giving the impression that the robot is a machine rather than a real agent. Therefore, we propose a neural network architecture based on the Seq2Seq model that learns social behaviors from human-human interactions in an end-to-end manner. We adopted a generative adversarial network to prevent invalid pose sequences from occurring when generating long-term behavior. To verify the proposed method, experiments were performed using the humanoid robot Pepper in a simulated environment. Because it is difficult to determine success or failure in social behavior generation, we propose new metrics to calculate the difference between the generated behavior and the ground-truth behavior. We used these metrics to show how different network architectural choices affect the performance of behavior generation, and we compared the performance of learning multiple behaviors and that of learning a single behavior. We expect that our proposed method can be used not only with home service robots, but also for guide robots, delivery robots, educational robots, and virtual robots, enabling the users to enjoy and effectively interact with the robots.