Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoders

TL;DR

This paper introduces noise-robust speech coding using vector-quantized autoencoders, improving performance in noisy environments and outperforming models trained solely on clean speech.

Contribution

It develops compressor-enhancer autoencoders based on VQ-VAE with WaveRNN decoders, enhancing noise robustness in speech coding.

Findings

Enhanced noise robustness in speech coding models

Compressor-enhancer models outperform pure compressor models in noisy conditions

Models trained on both clean and noisy speech perform better

Abstract

Audio codecs based on discretized neural autoencoders have recently been developed and shown to provide significantly higher compression levels for comparable quality speech output. However, these models are tightly coupled with speech content, and produce unintended outputs in noisy conditions. Based on VQ-VAE autoencoders with WaveRNN decoders, we develop compressor-enhancer encoders and accompanying decoders, and show that they operate well in noisy conditions. We also observe that a compressor-enhancer model performs better on clean speech inputs than a compressor model trained only on clean speech.