Branch Explorer: Leveraging Branching Narratives to Support Interactive 360{\deg} Video Viewing for Blind and Low Vision Users

TL;DR

Branch Explorer transforms 360-degree videos into branching narratives with audio guidance, enabling blind and low vision users to interactively explore immersive content and enhance their viewing experience.

Contribution

This paper introduces a novel system that converts 360-degree videos into accessible branching narratives using machine learning, specifically designed for BLV users.

Findings

Significantly improved user agency and engagement for BLV viewers.

Users developed personalized exploration strategies.

Effective generation of diverse narrative paths.

Abstract

360{\deg} videos enable users to freely choose their viewing paths, but blind and low vision (BLV) users are often excluded from this interactive experience. To bridge this gap, we present Branch Explorer, a system that transforms 360{\deg} videos into branching narratives -- stories that dynamically unfold based on viewer choices -- to support interactive viewing for BLV audiences. Our formative study identified three key considerations for accessible branching narratives: providing diverse branch options, ensuring coherent story progression, and enabling immersive navigation among branches. To address these needs, Branch Explorer employs a multi-modal machine learning pipeline to generate diverse narrative paths, allowing users to flexibly make choices at detected branching points and seamlessly engage with each storyline through immersive audio guidance. Evaluation with 12 BLV viewers showed that Branch Explorer significantly enhanced user agency and engagement in 360{\deg} video viewing. Users also developed personalized strategies for exploring 360{\deg} content. We further highlight implications for supporting accessible exploration of videos and virtual environments.