HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

TL;DR

This paper presents HiFi-GAN, a deep learning model that enhances degraded speech recordings by reducing noise and reverberation, achieving high perceptual quality and generalization across speakers and environments.

Contribution

Introduces HiFi-GAN, a novel end-to-end adversarial network utilizing multi-scale discriminators and deep feature matching for high-fidelity speech enhancement.

Findings

Outperforms state-of-the-art methods in objective metrics

Achieves superior subjective perceptual quality

Generalizes well to unseen speakers and environments

Abstract

Real-world audio recordings are often degraded by factors such as noise, reverberation, and equalization distortion. This paper introduces HiFi-GAN, a deep learning method to transform recorded speech to sound as though it had been recorded in a studio. We use an end-to-end feed-forward WaveNet architecture, trained with multi-scale adversarial discriminators in both the time domain and the time-frequency domain. It relies on the deep feature matching losses of the discriminators to improve the perceptual quality of enhanced speech. The proposed model generalizes well to new speakers, new speech content, and new environments. It significantly outperforms state-of-the-art baseline methods in both objective and subjective experiments.