Speech Denoising with Deep Feature Losses

TL;DR

This paper introduces a deep learning method for speech denoising that processes raw audio with a novel deep feature loss, outperforming existing techniques especially in challenging noisy environments.

Contribution

It proposes a fully-convolutional network trained with a deep feature loss based on an auxiliary network, achieving superior speech quality and noise reduction over prior methods.

Findings

Outperforms state-of-the-art in objective speech quality metrics

Achieves better perceptual scores in human listener tests

Particularly effective on highly noisy data

Abstract

We present an end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly. Given input audio containing speech corrupted by an additive background signal, the system aims to produce a processed signal that contains only the speech content. Recent approaches have shown promising results using various deep network architectures. In this paper, we propose to train a fully-convolutional context aggregation network using a deep feature loss. That loss is based on comparing the internal feature activations in a different network, trained for acoustic environment detection and domestic audio tagging. Our approach outperforms the state-of-the-art in objective speech quality metrics and in large-scale perceptual experiments with human listeners. It also outperforms an identical network trained using traditional regression losses. The advantage of the new approach is particularly pronounced for the hardest data with the most intrusive background noise, for which denoising is most needed and most challenging.