Applying General Turn-taking Models to Conversational Human-Robot Interaction

TL;DR

This paper explores the use of advanced turn-taking models, TurnGPT and VAP, to enhance the naturalness of human-robot conversations, demonstrating improved interaction quality and reduced delays.

Contribution

It introduces the novel application of general turn-taking models to HRI, trained on human dialogue data without domain-specific fine-tuning.

Findings

Participants preferred the proposed system over baseline.

The system reduced response delays.

It decreased interruptions during interactions.

Abstract

Turn-taking is a fundamental aspect of conversation, but current Human-Robot Interaction (HRI) systems often rely on simplistic, silence-based models, leading to unnatural pauses and interruptions. This paper investigates, for the first time, the application of general turn-taking models, specifically TurnGPT and Voice Activity Projection (VAP), to improve conversational dynamics in HRI. These models are trained on human-human dialogue data using self-supervised learning objectives, without requiring domain-specific fine-tuning. We propose methods for using these models in tandem to predict when a robot should begin preparing responses, take turns, and handle potential interruptions. We evaluated the proposed system in a within-subject study against a traditional baseline system, using the Furhat robot with 39 adults in a conversational setting, in combination with a large language model for autonomous response generation. The results show that participants significantly prefer the proposed system, and it significantly reduces response delays and interruptions.