A Neural Denoising Vocoder for Clean Waveform Generation from Noisy Mel-Spectrogram based on Amplitude and Phase Predictions

TL;DR

This paper introduces a neural denoising vocoder that reconstructs clean speech waveforms from noisy mel-spectrograms by predicting and enhancing amplitude and phase spectra, achieving state-of-the-art results.

Contribution

It presents a novel two-component neural vocoder combining spectrum prediction and enhancement modules for denoising from noisy mel-spectrograms.

Findings

Achieves state-of-the-art performance on VoiceBank+DEMAND dataset.

Performs comparably to advanced speech enhancement methods despite limited phase information.

Operates effectively at the spectral domain level using existing backbone models.

Abstract

This paper proposes a novel neural denoising vocoder that can generate clean speech waveforms from noisy mel-spectrograms. The proposed neural denoising vocoder consists of two components, i.e., a spectrum predictor and a enhancement module. The spectrum predictor first predicts the noisy amplitude and phase spectra from the input noisy mel-spectrogram, and subsequently the enhancement module recovers the clean amplitude and phase spectrum from noisy ones. Finally, clean speech waveforms are reconstructed through inverse short-time Fourier transform (iSTFT). All operations are performed at the frame-level spectral domain, with the APNet vocoder and MP-SENet speech enhancement model used as the backbones for the two components, respectively. Experimental results demonstrate that our proposed neural denoising vocoder achieves state-of-the-art performance compared to existing neural vocoders on the VoiceBank+DEMAND dataset. Additionally, despite the lack of phase information and partial amplitude information in the input mel-spectrogram, the proposed neural denoising vocoder still achieves comparable performance with the serveral advanced speech enhancement methods.