Improving X-Codec-2.0 for Multi-Lingual Speech: 25 Hz Latent Rate and 24 kHz Sampling

TL;DR

This paper enhances X-Codec-2.0 by reducing the latent rate to 25 Hz and increasing the sampling rate to 24 kHz, leading to better efficiency and perceptual quality in multilingual speech compression.

Contribution

Introducing a simple modification with pooling and increased decoder hop size to improve X-Codec-2.0's efficiency and audio fidelity without changing the core architecture.

Findings

Achieved a 0.29 MOS improvement over the baseline.

Attained the best performance among codecs at 25 Hz.

Demonstrated improved efficiency and perceptual quality.

Abstract

X-Codec-2.0 has shown strong performance in neural audio compression and multilingual speech modeling, operating at a 50 Hz latent rate and a 16 kHz sampling rate using frozen HuBERT features. While effective, this configuration limits temporal efficiency and audio fidelity. In this work, we explore a simple and effective modification by introducing additional pooling and increasing the decoder hop size. This reduces the latent rate from 50 Hz to 25 Hz and simultaneously raises the output sampling rate from 16 kHz to 24 kHz, improving efficiency and perceptual quality without altering the core architecture. Evaluated on the multilingual Common Voice 17 test set, the proposed configuration achieves a 0.29 MOS improvement over the original X-Codec-2.0 baseline based on UTMOSv2, and attains the best reported performance among all codecs operating at 25 Hz. The source code, checkpoints, and generation comparisons are released at \href{https://huggingface.co/Scicom-intl/xcodec2-25TPS-24k}{https://huggingface.co/Scicom-intl/xcodec2-25TPS-24k}.