Storycaster: An AI System for Immersive Room-Based Storytelling

TL;DR

Storycaster is an AI-powered CAVE system that transforms physical rooms into immersive, interactive storytelling environments by integrating live camera feeds, object editing, and co-creative narration with generated multimedia content.

Contribution

It introduces a novel AI system that combines room-awareness, object editing, and narrative guidance to enable dynamic, immersive storytelling in physical spaces.

Findings

Participants found the system highly immersive and engaging.

Narrator and audio significantly enhanced user experience.

Identified areas for improvement in latency and image resolution.

Abstract

While Cave Automatic Virtual Environment (CAVE) systems have long enabled room-scale virtual reality and various kinds of interactivity, their content has largely remained predetermined. We present \textit{Storycaster}, a generative AI CAVE system that transforms physical rooms into responsive storytelling environments. Unlike headset-based VR, \textit{Storycaster} preserves spatial awareness, using live camera feeds to augment the walls with cylindrical projections, allowing users to create worlds that blend with their physical surroundings. Additionally, our system enables object-level editing, where physical items in the room can be transformed to their virtual counterparts in a story. A narrator agent guides participants, enabling them to co-create stories that evolve in response to voice commands, with each scene enhanced by generated ambient audio, dialogue, and imagery. Participants in our study ($n=13$) found the system highly immersive and engaging, with narrator and audio most impactful, while also highlighting areas for improvement in latency and image resolution.