Real Time Speech Enhancement in the Waveform Domain

TL;DR

This paper introduces a real-time, causal waveform-based speech enhancement model that effectively reduces diverse background noises and reverb, optimized for CPU execution, with improved generalization through waveform data augmentation.

Contribution

A novel causal waveform-domain speech enhancement model that operates in real-time on CPU and incorporates waveform data augmentation for better performance and generalization.

Findings

Matches state-of-the-art performance on benchmarks

Effective at removing stationary and non-stationary noise

Operates in real-time on a laptop CPU

Abstract

We present a causal speech enhancement model working on the raw waveform that runs in real-time on a laptop CPU. The proposed model is based on an encoder-decoder architecture with skip-connections. It is optimized on both time and frequency domains, using multiple loss functions. Empirical evidence shows that it is capable of removing various kinds of background noise including stationary and non-stationary noises, as well as room reverb. Additionally, we suggest a set of data augmentation techniques applied directly on the raw waveform which further improve model performance and its generalization abilities. We perform evaluations on several standard benchmarks, both using objective metrics and human judgements. The proposed model matches state-of-the-art performance of both causal and non causal methods while working directly on the raw waveform.