Whispers of the Butterfly: A Research-through-Design Exploration of In-Situ Conversational AI Guidance in Large-Scale Outdoor MR Exhibitions

TL;DR

This paper explores the design and deployment of an in-situ conversational AI guide in large-scale outdoor MR art exhibitions, examining how it influences visitor experience and interaction dynamics.

Contribution

It introduces Dream-Butterfly, a novel embodied conversational AI for outdoor MR exhibitions, and provides empirical insights into its impact on visitor engagement and guidance roles.

Findings

AI guide reshapes explanation access and perceived responsiveness.

Visitors negotiate responsibility among staff, AI, and themselves.

Design implications for human-AI guiding roles in outdoor MR environments.

Abstract

Large-scale outdoor mixed reality (MR) art exhibitions distribute curated virtual works across open public spaces, but interpretation rarely scales without turning exploration into a scripted tour. Through Research-through-Design, we created Dream-Butterfly, an in-situ conversational AI docent embodied as a small non-human companion that visitors summon for multilingual, exhibition-grounded explanations. We deployed Dream-Butterfly in a large-scale outdoor MR exhibition at a public university campus in southern China, and conducted an in-the-wild between-subject study (N=24) comparing a primarily human-led tour with an AI-led tour while keeping staff for safety in both conditions. Combining questionnaires and semi-structured interviews, we characterize how shifting the primary explanation channel reshapes explanation access, perceived responsiveness, immersion, and workload, and how visitors negotiate responsibility handoffs among staff, the AI guide, and themselves. We distill transferable design implications for configuring mixed human-AI guiding roles and embodying conversational agents in mobile, safety-constrained outdoor MR exhibitions.