Facilitating Visual Media Exploration for Blind and Low Vision Users through AI-Powered Interactive Storytelling

TL;DR

This paper introduces AI-powered interactive storytelling techniques to enhance visual media exploration for blind and low vision users, improving engagement and comprehension within a coherent narrative framework.

Contribution

It presents a novel paradigm and three specific techniques for integrating interactive storytelling with visual media exploration for BLV users.

Findings

Hierarchical Narrative supports multi-level photo exploration

Parallel Narrative enables synchronized video comments access

Branching Narrative allows immersive 360° video navigation

Abstract

Empowering blind and low vision (BLV) users to explore visual media improves content comprehension, strengthens user agency, and fulfills diverse information needs. However, most existing tools separate exploration from the main narration, which disrupts the narrative flow, increases cognitive load, and limits deep engagement with visual media. To address these challenges, my PhD research introduces the paradigm of AI-powered interactive storytelling, which leverages AI to generate interactive narratives, enabling BLV users to explore visual media within a coherent storytelling experience. I have operationalized this paradigm through three techniques: (1) Hierarchical Narrative, which supports photo-collection exploration at different levels of detail; (2) Parallel Narrative, which provides seamless access to time-synced video comments; and (3) Branching Narrative, which enables immersive navigation of 360{\deg} videos. Together, these techniques demonstrate that AI-powered interactive storytelling can effectively balance user agency with narrative coherence across diverse media formats. My future work will advance this paradigm by enabling more personalized and expressive storytelling experiences for BLV audiences.