From Temporal to Spatial: Designing Spatialized Interactions with Segmented-audios in Immersive Environments for Active Engagement with Performing Arts Intangible Cultural Heritage

TL;DR

This paper introduces a Virtual Reality system that transforms passive listening of traditional performance art into active, spatially-interactive experiences, enhancing engagement with intangible cultural heritage like Peking opera.

Contribution

The paper presents a novel spatialized segmented-audio VR system that converts temporal music into spatial interactions, supported by co-design, prototyping, and user testing with traditional artforms.

Findings

Two distinct interaction patterns identified: Progressive and Adaptive.

SISA system effectively promotes active engagement with auditory ICH.

Spatialized audio clustering enhances user interaction and understanding.

Abstract

Performance artforms like Peking opera face transmission challenges due to the extensive passive listening required to understand their nuance. To create engaging forms of experiencing auditory Intangible Cultural Heritage (ICH), we designed a spatial interaction-based segmented-audio (SISA) Virtual Reality system that transforms passive ICH experiences into active ones. We undertook: (1) a co-design workshop with seven stakeholders to establish design requirements, (2) prototyping with five participants to validate design elements, and (3) user testing with 16 participants exploring Peking Opera. We designed transformations of temporal music into spatial interactions by cutting sounds into short audio segments, applying t-SNE algorithm to cluster audio segments spatially. Users navigate through these sounds by their similarity in audio property. Analysis revealed two distinct interaction patterns (Progressive and Adaptive), and demonstrated SISA's efficacy in facilitating active auditory ICH engagement. Our work illuminates the design process for enriching traditional performance artform using spatially-tuned forms of listening.