Vector Quantized Diffusion Model Based Speech Bandwidth Extension

TL;DR

This paper presents a novel speech bandwidth extension method using vector quantized diffusion models to enhance high-frequency speech details, resulting in improved speech quality and intelligibility.

Contribution

It introduces the first NAC-based BWE approach leveraging discrete features with diffusion models, combining advanced neural codecs and Mamba-2 for superior reconstruction.

Findings

Outperforms existing methods in log-spectral distance

Achieves higher ViSQOL scores

Enhances speech naturalness and intelligibility

Abstract

Recent advancements in neural audio codec (NAC) unlock new potential in audio signal processing. Studies have increasingly explored leveraging the latent features of NAC for various speech signal processing tasks. This paper introduces the first approach to speech bandwidth extension (BWE) that utilizes the discrete features obtained from NAC. By restoring high-frequency details within highly compressed discrete tokens, this approach enhances speech intelligibility and naturalness. Based on Vector Quantized Diffusion, the proposed framework combines the strengths of advanced NAC, diffusion models, and Mamba-2 to reconstruct high-frequency speech components. Extensive experiments demonstrate that this method exhibits superior performance across both log-spectral distance and ViSQOL, significantly improving speech quality.