Improving Networked Music Performance Systems Using Application-Network Collaboration

TL;DR

This paper presents a novel integrated approach combining application and network collaboration, using SDN principles, to maintain ultra-low latency in Networked Music Performance systems, ensuring synchronization within the 25ms threshold.

Contribution

It introduces a dynamic, integrated system that adjusts both network routing and audio processing in real-time to meet strict delay constraints in NMP systems.

Findings

The system effectively maintains end-to-end delay below 25ms in emulated environments.

Dynamic collaboration between application and network improves delay management.

Prototype demonstrates practical feasibility of the approach.

Abstract

Networked Music Performance (NMP) systems involve musicians located in different places who perform music while staying synchronized via the Internet. The maximum end-to-end delay in NMP applications is called Ensemble Performance Threshold (EPT) and should be less than 25 milliseconds. Due to this constraint, NMPs require ultra-low delay solutions for audio coding, network transmission, relaying and decoding, each one a challenging task on its own. There are two directions for study in the related work referring to the NMP systems. From the audio perspective, researchers experiment on low-delay encoders and transmission patterns, aiming to reduce the processing delay of the audio transmission, but they ignore the network performance. On the other hand, network-oriented researchers try to reduce the network delay, which contributes to reduced end-to-end delay. In our proposed approach, we introduce an integration of dynamic audio and network configuration to satisfy the EPT constraint. The basic idea is that the major components participating in an NMP system the application and the network interact during the live music performance. As the network delay increases, the network tries to equalize it by modifying the routing behavior using Software Defined Networking principles. If the network delay exceeds a maximum affordable threshold, the network reacts by informing the application to change the audio processing pattern to overcome the delay increase, resulting in below EPT end-to-end delay. A full prototype of the proposed system was implemented and extensively evaluated in an emulated environment.