On Improving Error Resilience of Neural End-to-End Speech Coders

TL;DR

This paper enhances the error resilience of neural end-to-end speech codecs by integrating a low complexity network for codebook index prediction and an in-band FEC, significantly improving robustness against packet losses.

Contribution

It introduces novel methods to improve packet loss robustness in neural speech codecs, including a codebook index prediction network and an in-band FEC at low bitrate.

Findings

Coupling PLC and FEC significantly improves robustness.

Proposed methods outperform baseline in both subjective and objective tests.

Low bitrate FEC adds minimal additional bitrate while enhancing error resilience.

Abstract

Error resilient tools like Packet Loss Concealment (PLC) and Forward Error Correction (FEC) are essential to maintain a reliable speech communication for applications like Voice over Internet Protocol (VoIP), where packets are frequently delayed and lost. In recent times, end-to-end neural speech codecs have seen a significant rise, due to their ability to transmit speech signal at low bitrates but few considerations were made about their error resilience in a real system. Recently introduced Neural End-to-End Speech Codec (NESC) can reproduce high quality natural speech at low bitrates. We extend its robustness to packet losses by adding a low complexity network to predict the codebook indices in latent space. Furthermore, we propose a method to add an in-band FEC at an additional bitrate of 0.8 kbps. Both subjective and objective assessment indicate the effectiveness of proposed methods, and demonstrate that coupling PLC and FEC provide significant robustness against packet losses.