A Deep Learning Approach for Low-Latency Packet Loss Concealment of Audio Signals in Networked Music Performance Applications

TL;DR

This paper presents a deep learning-based method for real-time prediction and concealment of lost audio packets in networked music performance applications, aiming to improve audio quality under strict latency constraints.

Contribution

It introduces a novel deep learning approach for low-latency packet loss concealment specifically tailored for real-time networked music performance scenarios.

Findings

Effective real-time packet loss prediction demonstrated

Significant reduction in audio glitches observed

Improved perceived audio quality in tests

Abstract

Networked Music Performance (NMP) is envisioned as a potential game changer among Internet applications: it aims at revolutionizing the traditional concept of musical interaction by enabling remote musicians to interact and perform together through a telecommunication network. Ensuring realistic conditions for music performance, however, constitutes a significant engineering challenge due to extremely strict requirements in terms of audio quality and, most importantly, network delay. To minimize the end-to-end delay experienced by the musicians, typical implementations of NMP applications use un-compressed, bidirectional audio streams and leverage UDP as transport protocol. Being connection less and unreliable,audio packets transmitted via UDP which become lost in transit are not re-transmitted and thus cause glitches in the receiver audio playout. This article describes a technique for predicting lost packet content in real-time using a deep learning approach. The ability of concealing errors in real time can help mitigate audio impairments caused by packet losses, thus improving the quality of audio playout in real-world scenarios.