Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents

TL;DR

This paper explores how conversational fillers like gestures and verbal cues can reduce perceived response delays in VR-based LLM virtual agents, especially under high latency, and provides an open-source deployment pipeline.

Contribution

It introduces an effective method using conversational fillers to mitigate response delays and offers an open-source pipeline for deploying such agents in virtual environments.

Findings

Latency above 4 seconds degrades user experience

Conversational fillers improve perceived response time in high-delay scenarios

Open-source pipeline facilitates deployment of virtual agents in VR

Abstract

We investigated the challenges of mitigating response delays in free-form conversations with virtual agents powered by Large Language Models (LLMs) within Virtual Reality (VR). For this, we used conversational fillers, such as gestures and verbal cues, to bridge delays between user input and system responses and evaluate their effectiveness across various latency levels and interaction scenarios. We found that latency above 4 seconds degrades quality of experience, while natural conversational fillers improve perceived response time, especially in high-delay conditions. Our findings provide insights for practitioners and researchers to optimize user engagement whenever conversational systems' responses are delayed by network limitations or slow hardware. We also contribute an open-source pipeline that streamlines deploying conversational agents in virtual environments.