Whispers of the Butterfly: A Research-through-Design Exploration of In-Situ Conversational AI Guidance in Large-Scale Outdoor MR Exhibitions
Dongyijie Primo Pan, Shuyue Li, Yawei Zhao, Junkun Long, Hao Li, Pan Hui

TL;DR
This paper explores the design and deployment of an in-situ conversational AI guide in large-scale outdoor MR art exhibitions, examining how it influences visitor experience and interaction dynamics.
Contribution
It introduces Dream-Butterfly, a novel embodied conversational AI for outdoor MR exhibitions, and provides empirical insights into its impact on visitor engagement and guidance roles.
Findings
AI guide reshapes explanation access and perceived responsiveness.
Visitors negotiate responsibility among staff, AI, and themselves.
Design implications for human-AI guiding roles in outdoor MR environments.
Abstract
Large-scale outdoor mixed reality (MR) art exhibitions distribute curated virtual works across open public spaces, but interpretation rarely scales without turning exploration into a scripted tour. Through Research-through-Design, we created Dream-Butterfly, an in-situ conversational AI docent embodied as a small non-human companion that visitors summon for multilingual, exhibition-grounded explanations. We deployed Dream-Butterfly in a large-scale outdoor MR exhibition at a public university campus in southern China, and conducted an in-the-wild between-subject study (N=24) comparing a primarily human-led tour with an AI-led tour while keeping staff for safety in both conditions. Combining questionnaires and semi-structured interviews, we characterize how shifting the primary explanation channel reshapes explanation access, perceived responsiveness, immersion, and workload, and how…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVirtual Reality Applications and Impacts · Social Robot Interaction and HRI · Innovative Human-Technology Interaction
