Jigsaw: Authoring Immersive Storytelling Experiences with Augmented Reality and Internet of Things

TL;DR

Jigsaw is a user-friendly system that enables beginners to create and experience immersive AR stories by integrating virtual and physical elements using IoT devices, addressing technical complexity and enhancing engagement.

Contribution

The paper introduces Jigsaw, a novel platform that simplifies immersive storytelling creation with AR and IoT, making it accessible to non-experts.

Findings

Participants successfully created immersive stories.

Stories were found to be deeply engaging.

Sensory overload was a notable challenge.

Abstract

Augmented Reality (AR) presents new opportunities for immersive storytelling. However, this immersiveness faces two main hurdles. First, AR's immersive quality is often confined to visual elements, such as pixels on a screen. Second, crafting immersive narratives is complex and generally beyond the reach of amateurs due to the need for advanced technical skills. We introduce Jigsaw, a system that empowers beginners to both experience and craft immersive stories, blending virtual and physical elements. Jigsaw uniquely combines mobile AR with readily available Internet-of-things (IoT) devices. We conducted a qualitative study with 20 participants to assess Jigsaw's effectiveness in both consuming and creating immersive narratives. The results were promising: participants not only successfully created their own immersive stories but also found the playback of three such stories deeply engaging. However, sensory overload emerged as a significant challenge in these experiences. We discuss design trade-offs and considerations for future endeavors in immersive storytelling involving AR and IoT.