When to Say "Hi" -- Learn to Open a Conversation with an in-the-wild Dataset

TL;DR

This paper presents the Interaction Initiation System (IIS), a machine learning approach trained on in-the-wild data to detect when and who should initiate conversation in socially interactive agents, improving interaction smoothness.

Contribution

The paper introduces IIS, a novel system trained on real-world data to identify conversation openings, advancing social agent interaction capabilities.

Findings

IIS accurately detects greeting periods.

The system identifies the interaction opener.

Validated with 201 user interactions.

Abstract

The social capabilities of socially interactive agents (SIA) are a key to successful and smooth interactions between the user and the SIA. A successful start of the interaction is one of the essential factors for satisfying SIA interactions. For a service and information task in which the SIA helps with information, e.g. about the location, it is an important skill to master the opening of the conversation and to recognize which interlocutor opens the conversation and when. We are therefore investigating the extent to which the opening of the conversation can be trained using the user's body language as an input for machine learning to ensure smooth conversation starts for the interaction. In this paper we propose the Interaction Initiation System (IIS) which we developed, trained and validated using an in-the-wild data set. In a field test at the Deutsches Museum Bonn, a Furhat robot from Furhat Robotics was used as a service and information point. Over the period of use we collected the data of \textit{N} = 201 single user interactions for the training of the algorithms. We can show that the IIS, achieves a performance that allows the conclusion that this system is able to determine the greeting period and the opener of the interaction.