Interactive Robot Learning of Gestures, Language and Affordances

TL;DR

This paper presents a model enabling robots to learn gestures, language, and affordances through interaction, combining environment exploration with human gesture recognition to improve human-robot collaboration.

Contribution

It introduces a unified probabilistic model that integrates affordance learning, language understanding, and gesture recognition for cognitive robots.

Findings

Robots can generalize learned environment knowledge to human actions.

Initial results show successful recognition of human gestures.

The model supports improved human-robot interaction.

Abstract

A growing field in robotics and Artificial Intelligence (AI) research is human-robot collaboration, whose target is to enable effective teamwork between humans and robots. However, in many situations human teams are still superior to human-robot teams, primarily because human teams can easily agree on a common goal with language, and the individual members observe each other effectively, leveraging their shared motor repertoire and sensorimotor resources. This paper shows that for cognitive robots it is possible, and indeed fruitful, to combine knowledge acquired from interacting with elements of the environment (affordance exploration) with the probabilistic observation of another agent's actions. We propose a model that unites (i) learning robot affordances and word descriptions with (ii) statistical recognition of human gestures with vision sensors. We discuss theoretical motivations, possible implementations, and we show initial results which highlight that, after having acquired knowledge of its surrounding environment, a humanoid robot can generalize this knowledge to the case when it observes another agent (human partner) performing the same motor actions previously executed during training.