Sequential annotations for naturally-occurring HRI: first insights

TL;DR

This paper presents a new methodology for annotating natural human-robot interactions to improve conversational agents, using conversation analysis and multimodal data, exemplified by a Pepper robot in a library setting.

Contribution

It introduces a novel annotation practice based on theoretical insights, creating a publicly available corpus to enhance human-robot interaction models.

Findings

Developed a new annotation methodology for HRI

Created a corpus of naturally-occurring interactions

Provides insights into multimodal communication in HRI

Abstract

We explain the methodology we developed for improving the interactions accomplished by an embedded conversational agent, drawing from Conversation Analytic sequential and multimodal analysis. The use case is a Pepper robot that is expected to inform and orient users in a library. In order to propose and learn better interactive schema, we are creating a corpus of naturally-occurring interactions that will be made available to the community. To do so, we propose an annotation practice based on some theoretical underpinnings about the use of language and multimodal resources in human-robot interaction. CCS CONCEPTS $\bullet$ Computing methodologies $\rightarrow$ Discourse, dialogue and pragmatics; $\bullet$ Human-centered computing $\rightarrow$ Text input; HCI theory, concepts and models; Field studies.