Modeling Human Response to Robot Errors for Timely Error Detection

TL;DR

This paper explores how natural social responses to robot errors can be used to automatically detect errors in physical human-robot collaboration, demonstrating robustness across various tasks and error types.

Contribution

It introduces a method leveraging social responses for real-time error detection in non-social, physical human-robot interactions, expanding beyond prior social interaction focus.

Findings

Social responses effectively signal robot errors in real-time.

Method generalizes across different tasks and error types.

Robust error detection without detailed task specifications.

Abstract

In human-robot collaboration, robot errors are inevitable -- damaging user trust, willingness to work together, and task performance. Prior work has shown that people naturally respond to robot errors socially and that in social interactions it is possible to use human responses to detect errors. However, there is little exploration in the domain of non-social, physical human-robot collaboration such as assembly and tool retrieval. In this work, we investigate how people's organic, social responses to robot errors may be used to enable timely automatic detection of errors in physical human-robot interactions. We conducted a data collection study to obtain facial responses to train a real-time detection algorithm and a case study to explore the generalizability of our method with different task settings and errors. Our results show that natural social responses are effective signals for timely detection and localization of robot errors even in non-social contexts and that our method is robust across a variety of task contexts, robot errors, and user responses. This work contributes to robust error detection without detailed task specifications.