ARCADE: An Augmented Reality Display Environment for Multimodal Interaction with Conversational Agents

TL;DR

ARCADE is a spatial augmented reality system that enables natural, multimodal interaction with virtual conversational agents by creating an immersive, room-like illusion using optical-see-through displays, demonstrated in various dialogue scenarios.

Contribution

We introduce ARCADE, a novel augmented reality environment that displays virtual agents in a room-like setting for seamless multimodal interaction.

Findings

ARCADE effectively displays virtual agents in real space.

The system supports natural multimodal interaction.

Demonstrated in multiple dialogue scenarios.

Abstract

Making the interaction with embodied conversational agents accessible in a ubiquitous and natural manner is not only a question of the underlying software but also brings challenges in terms of the technical system that is used to display them. To this end, we present our spatial augmented reality system ARCADE, which can be utilized like a conventional monitor for displaying virtual agents as well as additional content. With its optical-see-through display, ARCADE creates the illusion of the agent being in the room similarly to a human. The applicability of our system is demonstrated in two different dialogue scenarios, which are included in the video accompanying this paper at https://youtu.be/9nH4c4Q-ooE.