Switchcodec: Adaptive residual-expert sparse quantization for high-fidelity neural audio coding

TL;DR

SwitchCodec introduces a dynamic neural audio codec utilizing residual expert vector quantization, enabling adaptive bitrate and improved compression efficiency for diverse audio content.

Contribution

It proposes REVQ with dynamic expert routing and variable bitrate mechanisms, enhancing neural audio coding beyond fixed codebook approaches.

Findings

Outperforms existing baselines on objective metrics.

Achieves superior subjective listening test results.

Supports multi-bitrate operation without retraining.

Abstract

Recent neural audio compression models often rely on residual vector quantization for high-fidelity coding, but using a fixed number of per-frame codebooks is suboptimal for the wide variability of audio content-especially for signals that are either very simple or highly complex. To address this limitation, we propose SwitchCodec, a neural audio codec based on Residual Experts Vector Quantization (REVQ). REVQ combines a shared quantizer with dynamically routed expert quantizers that are activated according to the input audio, decoupling bitrate from codebook capacity and improving compression efficiency. This design ensures full training and utilization of each quantizer. In addition, a variable-bitrate mechanism adjusts the number of active expert quantizers at inference, enabling multi-bitrate operation without retraining. Experiments demonstrate that SwitchCodec surpasses existing baselines on both objective metrics and subjective listening tests.