Why Report Failed Interactions With Robots?! Towards Vignette-based Interaction Quality

TL;DR

This paper advocates for using ethnographic vignettes to better document and communicate failures in human-robot interactions, especially those rarely reported, to improve transparency and understanding of system limitations.

Contribution

It introduces a novel methodology using ethnographic vignettes in HRI research to highlight and communicate failures more effectively.

Findings

Vignettes help communicate failures from multiple perspectives.

They promote transparency about robot capabilities.

Vignettes can augment existing evaluation methods.

Abstract

Although the quality of human-robot interactions has improved with the advent of LLMs, there are still various factors that cause systems to be sub-optimal when compared to human-human interactions. The nature and criticality of failures are often dependent on the context of the interaction and so cannot be generalized across the wide range of scenarios and experiments which have been implemented in HRI research. In this work we propose the use of a technique overlooked in the field of HRI, ethnographic vignettes, to clearly highlight these failures, particularly those that are rarely documented. We describe the methodology behind the process of writing vignettes and create our own based on our personal experiences with failures in HRI systems. We emphasize the strength of vignettes as the ability to communicate failures from a multi-disciplinary perspective, promote transparency about the capabilities of robots, and document unexpected behaviours which would otherwise be omitted from research reports. We encourage the use of vignettes to augment existing interaction evaluation methods.