Dynamic Real-Time Ambisonics Order Adaptation for Immersive Networked Music Performances

TL;DR

This paper introduces a real-time adaptive Ambisonics system that dynamically adjusts audio encoding order based on network conditions to ensure immersive and reliable networked music performances.

Contribution

It presents a novel method for real-time Ambisonics order adaptation that balances audio quality and network reliability in immersive remote music sessions.

Findings

Adaptive approach maintains audio quality under bandwidth constraints.

System reduces audio dropouts during network impairments.

User evaluations show improved experience with adaptive strategy.

Abstract

Advanced remote applications such as Networked Music Performance (NMP) require solutions to guarantee immersive real-world-like interaction among users. Therefore, the adoption of spatial audio formats, such as Ambisonics, is fundamental to let the user experience an immersive acoustic scene. The accuracy of the sound scene reproduction increases with the order of the Ambisonics enconding, resulting in an improved immersivity at the cost of a greater number of audio channels, which in turn escalates both bandwidth requirements and susceptibility to network impairments (e.g., latency, jitter, and packet loss). These factors pose a significant challenge for interactive music sessions, which demand high spatial fidelity and low end-to-end delay. We propose a real-time adaptive higher-order Ambisonics strategy that continuously monitors network throughput and dynamically scales the Ambisonics order. When available bandwidth drops below a preset threshold, the order is lowered to prevent audio dropouts; it then reverts to higher orders once conditions recover, thus balancing immersion and reliability. A MUSHRA-based evaluation indicates that this adaptive approach is promising to guarantee user experience in bandwidth-limited NMP scenarios.