SE-MelGAN -- Speaker Agnostic Rapid Speech Enhancement

TL;DR

This paper introduces SE-MelGAN, a robust, speaker-agnostic speech enhancement model based on GANs that generalizes well to unseen noises, improves quality and speed over previous methods, and operates in real-time without hardware optimization.

Contribution

It adapts MelGAN's robustness to speech enhancement, demonstrating multi-speaker generalization, noise robustness, and faster convergence without model modifications.

Findings

Outperforms SEGAN in quality and speed.

Operates at over 100x real-time speed on GPU.

Handles unseen background noises effectively.

Abstract

Recent advancement in Generative Adversarial Networks in speech synthesis domain[3],[2] have shown, that it's possible to train GANs [8] in a reliable manner for high quality coherent waveform generation from mel-spectograms. We propose that it is possible to transfer the MelGAN's [3] robustness in learning speech features to speech enhancement and noise reduction domain without any model modification tasks. Our proposed method generalizes over multi-speaker speech dataset and is able to robustly handle unseen background noises during the inference. Also, we show that by increasing the batch size for this particular approach not only yields better speech results, but generalizes over multi-speaker dataset easily and leads to faster convergence. Additionally, it outperforms previous state of the art GAN approach for speech enhancement SEGAN [5] in two domains: 1. quality ; 2. speed. Proposed method runs at more than 100x faster than realtime on GPU and more than 2x faster than real time on CPU without any hardware optimization tasks, right at the speed of MelGAN [3].