CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram

TL;DR

CleanUNet 2 introduces a hybrid speech denoising approach combining waveform and spectrogram models, leading to superior denoising performance through a two-stage framework inspired by speech synthesis techniques.

Contribution

It presents a novel two-stage speech denoising model that integrates waveform and spectrogram denoisers, enhancing denoising effectiveness over existing methods.

Findings

Outperforms previous speech denoising methods in objective metrics

Achieves better subjective audio quality in evaluations

Demonstrates the effectiveness of combining waveform and spectrogram models

Abstract

In this work, we present CleanUNet 2, a speech denoising model that combines the advantages of waveform denoiser and spectrogram denoiser and achieves the best of both worlds. CleanUNet 2 uses a two-stage framework inspired by popular speech synthesis methods that consist of a waveform model and a spectrogram model. Specifically, CleanUNet 2 builds upon CleanUNet, the state-of-the-art waveform denoiser, and further boosts its performance by taking predicted spectrograms from a spectrogram denoiser as the input. We demonstrate that CleanUNet 2 outperforms previous methods in terms of various objective and subjective evaluations.