Let's be friends! A rapport-building 3D embodied conversational agent for the Human Support Robot

TL;DR

This study explores how integrating a 3D embodied conversational agent capable of mirroring nonverbal cues with a human support robot can enhance user experience and rapport during interactions involving mobile manipulative tasks.

Contribution

We developed a novel integrated system combining an expressive ECA with a support robot, capable of real-time facial and head movement mirroring to improve human-robot interaction.

Findings

Preliminary results indicate improved user engagement.

Mirroring nonverbal cues enhances rapport-building.

System shows promise for future larger studies.

Abstract

Partial subtle mirroring of nonverbal behaviors during conversations (also known as mimicking or parallel empathy), is essential for rapport building, which in turn is essential for optimal human-human communication outcomes. Mirroring has been studied in interactions between robots and humans, and in interactions between Embodied Conversational Agents (ECAs) and humans. However, very few studies examine interactions between humans and ECAs that are integrated with robots, and none of them examine the effect of mirroring nonverbal behaviors in such interactions. Our research question is whether integrating an ECA able to mirror its interlocutor's facial expressions and head movements (continuously or intermittently) with a human-service robot will improve the user's experience with the support robot that is able to perform useful mobile manipulative tasks (e.g. at home). Our contribution is the complex integration of an expressive ECA, able to track its interlocutor's face, and to mirror his/her facial expressions and head movements in real time, integrated with a human support robot such that the robot and the agent are fully aware of each others', and of the users', nonverbals cues. We also describe a pilot study we conducted towards answering our research question, which shows promising results for our forthcoming larger user study.