iCub Knows Where You Look: Exploiting Social Cues for Interactive Object Detection Learning

TL;DR

This paper presents a social cue-based approach enabling the iCub robot to learn new objects through natural human interaction, leveraging joint attention and social signals to improve interactive object detection in dynamic environments.

Contribution

It introduces a novel method combining social cues like gaze and speech to facilitate robot learning of objects during interactive sessions.

Findings

Robot successfully learns new objects via social interaction.

Social cues improve the robustness of object detection.

System operates effectively amidst distractors.

Abstract

Performing joint interaction requires constant mutual monitoring of own actions and their effects on the other's behaviour. Such an action-effect monitoring is boosted by social cues and might result in an increasing sense of agency. Joint actions and joint attention are strictly correlated and both of them contribute to the formation of a precise temporal coordination. In human-robot interaction, the robot's ability to establish joint attention with a human partner and exploit various social cues to react accordingly is a crucial step in creating communicative robots. Along the social component, an effective human-robot interaction can be seen as a new method to improve and make the robot's learning process more natural and robust for a given task. In this work we use different social skills, such as mutual gaze, gaze following, speech and human face recognition, to develop an effective teacher-learner scenario tailored to visual object learning in dynamic environments. Experiments on the iCub robot demonstrate that the system allows the robot to learn new objects through a natural interaction with a human teacher in presence of distractors.