Using Social Cues to Recognize Task Failures for HRI: Overview, State-of-the-Art, and Future Directions

TL;DR

This paper reviews how social cues can be used by robots to recognize task failures in human-robot interaction, proposing a framework and research agenda for integrating behavioral science, HRI, and machine learning.

Contribution

It introduces a comprehensive framework and taxonomy for using social cues as feedback signals to detect task failures in robots, guiding future research and development.

Findings

Proposes a taxonomy of error detection based on self-awareness and social feedback.

Identifies three key research areas: social cues as feedback, failure recognition, and autonomous detection methods.

Provides practical recommendations for implementing social cue-based error detection in HRI systems.

Abstract

Robots that carry out tasks and interact in complex environments will inevitably commit errors. Error detection is thus an essential ability for robots to master to work efficiently and productively. People can leverage social feedback to get an indication of whether an action was successful or not. With advances in computing and artificial intelligence (AI), it is increasingly possible for robots to achieve a similar capability of collecting social feedback. In this work, we take this one step further and propose a framework for how social cues can be used as feedback signals to recognize task failures for human-robot interaction (HRI). Our proposed framework sets out a research agenda based on insights from the literature on behavioral science, human-robot interaction, and machine learning to focus on three areas: 1) social cues as feedback (from behavioral science), 2) recognizing task failures in robots (from HRI), and 3) approaches for autonomous detection of HRI task failures based on social cues (from machine learning). We propose a taxonomy of error detection based on self-awareness and social feedback. Finally, we provide recommendations for HRI researchers and practitioners interested in developing robots that detect task errors using human social cues. This article is intended for interdisciplinary HRI researchers and practitioners, where the third theme of our analysis provides more technical details aiming toward the practical implementation of these systems.