Improved Speech Enhancement with the Wave-U-Net

TL;DR

This paper demonstrates that Wave-U-Net, an end-to-end time-domain model originally for music source separation, effectively improves speech enhancement metrics on the VCTK dataset, with fewer layers needed than in music applications.

Contribution

It adapts Wave-U-Net for speech enhancement, showing improved metrics and reduced model complexity compared to its original use in music source separation.

Findings

Improved PESQ, CSIG, CBAK, COVL, SSNR metrics.

Fewer hidden layers needed for speech enhancement.

Potential for use as pre-processing for speech recognition.

Abstract

We study the use of the Wave-U-Net architecture for speech enhancement, a model introduced by Stoller et al for the separation of music vocals and accompaniment. This end-to-end learning method for audio source separation operates directly in the time domain, permitting the integrated modelling of phase information and being able to take large temporal contexts into account. Our experiments show that the proposed method improves several metrics, namely PESQ, CSIG, CBAK, COVL and SSNR, over the state-of-the-art with respect to the speech enhancement task on the Voice Bank corpus (VCTK) dataset. We find that a reduced number of hidden layers is sufficient for speech enhancement in comparison to the original system designed for singing voice separation in music. We see this initial result as an encouraging signal to further explore speech enhancement in the time-domain, both as an end in itself and as a pre-processing step to speech recognition systems.