Spatial Orchestra: Locomotion Music Instruments through Spatial Exploration

TL;DR

Spatial Orchestra introduces an accessible AR system where users create music through natural locomotion by stepping into color-coded sound bubbles, transforming spatial awareness into musical expression.

Contribution

It presents a novel AR-based method enabling users of all skill levels to produce music via simple physical movements in a spatial environment.

Findings

Users can effortlessly produce music by walking into virtual sound bubbles.

The system enhances spatial awareness and bodily expression through interactive musical exploration.

Participants experience a new form of musical performance using natural locomotion in AR.

Abstract

Spatial Orchestra demonstrates how easy it is to play musical instruments using basic input like natural locomotion, which is accessible to most. Unlike many musical instruments, our work allows individuals of all skill levels to effortlessly create music by walking into virtual bubbles. Our Augmented Reality experience involves interacting with ever-shifting sound bubbles that the user engages with by stepping into color-coded bubbles within the assigned area using a standalone AR headset. Each bubble corresponds to a cello note, and omits sound from the center of the bubble, and lets the user hear and express in spatial audio, effectively transforming participants into musicians. This interactive element enables users to explore the intersection of spatial awareness, musical rhythm that extends to bodily expression through playful movements and dance-like gestures within the bubble-filled environment. This unique experience illuminates the intricate relationship between spatial awareness and the art of musical performance.