Behavioral and Symbolic Fillers as Delay Mitigation for Embodied Conversational Agents in Virtual Reality

TL;DR

This study explores how behavioral and symbolic fillers during response delays in virtual reality ECAs affect user experience, finding behavioral fillers improve perception and presence, while symbolic indicators have limited positive impact.

Contribution

It introduces and evaluates behavioral and symbolic fillers as delay mitigation strategies for ECAs in virtual reality, demonstrating their effects on user perception and interaction quality.

Findings

Behavioral fillers increased perceived response time and presence.

Symbolic indicators led to more gaze aversion but did not enhance perception.

Most participants preferred behavioral fillers over symbolic indicators.

Abstract

When communicating with embodied conversational agents (ECAs) in virtual reality, there might be delays in the responses of the agents lasting several seconds, for example, due to more extensive computations of the answers when large language models are used. Such delays might lead to unnatural or frustrating interactions. In this paper, we investigate filler types to mitigate these effects and lead to a more positive experience and perception of the agent. In a within-subject study, we asked 24 participants to communicate with ECAs in virtual reality, comparing four strategies displayed during the delays: a multimodal behavioral filler consisting of conversational and gestural fillers, a base condition with only idle motions, and two symbolic indicators with progress bars, one embedded as a badge on the agent, the other one external and visualized as a thinking bubble. Our results indicate that the behavioral filler improved perceived response time, three subscales of presence, humanlikeness, and naturalness. Participants looked away from the face more often when symbolic indicators were displayed, but the visualizations did not lead to a more positive impression of the agent or to increased presence. The majority of participants preferred the behavioral fillers, only 12.5% and 4.2% favored the symbolic embedded and external conditions, respectively.