Robot Error Awareness Through Human Reactions: Implementation, Evaluation, and Recommendations

TL;DR

This paper presents a proactive robot error detection system that leverages social signals and user feedback, demonstrating faster detection and higher user approval compared to traditional reactive methods.

Contribution

The study introduces a novel proactive error detection system combining behavioral signals, feedback, and context, with empirical evaluation showing improved performance over reactive approaches.

Findings

System reliably detects errors using social signals.

Detects errors faster than reactive methods.

Users perceive the proactive system more favorably.

Abstract

Effective error detection is crucial to prevent task disruption and maintain user trust. Traditional methods often rely on task-specific models or user reporting, which can be inflexible or slow. Recent research suggests social signals, naturally exhibited by users in response to robot errors, can enable more flexible, timely error detection. However, most studies rely on post hoc analysis, leaving their real-time effectiveness uncertain and lacking user-centric evaluation. In this work, we developed a proactive error detection system that combines user behavioral signals (facial action units and speech), user feedback, and error context for automatic error detection. In a study (N = 28), we compared our proactive system to a status quo reactive approach. Results show our system 1) reliably and flexibly detects error, 2) detects errors faster than the reactive approach, and 3) is perceived more favorably by users than the reactive one. We discuss recommendations for enabling robot error awareness in future HRI systems.