Rate-Aware Learned Speech Compression

TL;DR

This paper introduces a rate-aware neural speech compression method that replaces traditional quantizers with an entropy model and uses advanced neural blocks, achieving state-of-the-art rate-distortion performance and significant bitrate savings.

Contribution

It proposes a novel rate-aware compression scheme with an entropy model and enhanced neural blocks, improving RD performance and training simplicity over prior neural codecs.

Findings

Achieves 53.51% BD-Rate bitrate saving.

Gains of 0.26 BD-VisQol and 0.44 BD-PESQ.

Outperforms existing neural speech codecs.

Abstract

The rapid rise of real-time communication and large language models has significantly increased the importance of speech compression. Deep learning-based neural speech codecs have outperformed traditional signal-level speech codecs in terms of rate-distortion (RD) performance. Typically, these neural codecs employ an encoder-quantizer-decoder architecture, where audio is first converted into latent code feature representations and then into discrete tokens. However, this architecture exhibits insufficient RD performance due to two main drawbacks: (1) the inadequate performance of the quantizer, challenging training processes, and issues such as codebook collapse; (2) the limited representational capacity of the encoder and decoder, making it difficult to meet feature representation requirements across various bitrates. In this paper, we propose a rate-aware learned speech compression scheme that replaces the quantizer with an advanced channel-wise entropy model to improve RD performance, simplify training, and avoid codebook collapse. We employ multi-scale convolution and linear attention mixture blocks to enhance the representational capacity and flexibility of the encoder and decoder. Experimental results demonstrate that the proposed method achieves state-of-the-art RD performance, obtaining 53.51% BD-Rate bitrate saving in average, and achieves 0.26 BD-VisQol and 0.44 BD-PESQ gains.