Speech Enhancement via Deep Spectrum Image Translation Network

TL;DR

This paper introduces a novel deep learning-based speech enhancement method using a VGG19-UNet architecture with perceptually-modified spectrum images, significantly improving speech quality and intelligibility in noisy environments.

Contribution

It proposes a new VGG19-UNet architecture with perceptually-inspired spectrum images for improved speech enhancement in unseen noise conditions.

Findings

Outperforms existing methods in PESQ and ESTOI metrics

Effective in unseen noise environments

Utilizes perceptually-motivated spectrum representations

Abstract

Quality and intelligibility of speech signals are degraded under additive background noise which is a critical problem for hearing aid and cochlear implant users. Motivated to address this problem, we propose a novel speech enhancement approach using a deep spectrum image translation network. To this end, we suggest a new architecture, called VGG19-UNet, where a deep fully convolutional network known as VGG19 is embedded at the encoder part of an image-to-image translation network, i.e. U-Net. Moreover, we propose a perceptually-modified version of the spectrum image that is represented in Mel frequency and power-law non-linearity amplitude domains, representing good approximations of human auditory perception model. By conducting experiments on a real challenge in speech enhancement, i.e. unseen noise environments, we show that the proposed approach outperforms other enhancement methods in terms of both quality and intelligibility measures, represented by PESQ and ESTOI, respectively.