Audience Amplified: Virtual Audiences in Asynchronously Performed AR Theater

TL;DR

This paper presents a novel mobile AR theater experience featuring virtual audiences trained from real ones, enhancing social perception and engagement in asynchronous AR performances.

Contribution

It introduces a method to create AI-driven virtual audiences for AR theater, capturing human imperfections to simulate social presence in mobile augmented reality.

Findings

Virtual audiences increased perceived social presence.

Participants showed higher engagement with virtual audiences.

Some attendees preferred performances without virtual audiences.

Abstract

Audience reactions can considerably enhance live experiences; conversely, in anytime-anywhere augmented reality (AR) experiences, large crowds of people might not always be available to congregate. To get closer to simulating live events with large audiences, we created a mobile AR experience where users can wander around naturally and engage in AR theater with virtual audiences trained from real audiences using imitation learning. This allows us to carefully capture the essence of human imperfections and behavior in artificial intelligence (AI) audiences. The result is a novel mobile AR experience in which solitary AR users experience an augmented performance in a physical space with a virtual audience. Virtual dancers emerge from the surroundings, accompanied by a digitally simulated audience, to provide a community experience akin to immersive theater. In a pilot study, simulated human avatars were vastly preferred over just audience audio commentary. We subsequently engaged 20 participants as attendees of an AR dance performance, comparing a no-audience condition with a simulated audience of six onlookers. Through questionnaires and experience reports, we investigated user reactions and behavior. Our results demonstrate that the presence of virtual audience members caused attendees to perceive the performance as a social experience with increased interest and involvement in the event. On the other hand, for some attendees, the dance performances without the virtual audience evoked a stronger positive sentiment.