Speech Enhancement-assisted Voice Conversion in Noisy Environments

TL;DR

This paper introduces EStarGAN, a joint training speech enhancement and voice conversion system that improves the quality and robustness of converted speech in noisy environments, validated through experiments on Mandarin speech data.

Contribution

The paper presents a novel joint training approach combining speech enhancement and voice conversion using StarGAN, enhancing quality in noisy conditions.

Findings

EStarGAN's joint training improves speech quality.

System demonstrates robustness in unseen noisy environments.

Subjective tests confirm quality improvements.

Abstract

Numerous voice conversion (VC) techniques have been proposed for the conversion of voices among different speakers. Although good quality of the converted speech can be observed when VC is applied in a clean environment, the quality degrades drastically when the system is run in noisy conditions. In order to address this issue, we propose a novel speech enhancement (SE)-assisted VC system that utilizes the SE techniques for signal pre-processing, where the VC and SE components are optimized in an joint training strategy with the aim to provide high-quality converted speech signals. We adopt a popular model, StarGAN, as the VC component and thus call the combined system as EStarGAN. We test the proposed EStarGAN system using a Mandarin speech corpus. The experimental results first verified the effectiveness of joint training strategy used in EStarGAN. Moreover, EStarGAN demonstrated performance robustness in various unseen noisy environments. The subjective listening test results further showed that EStarGAN can improve the sound quality of speech signals converted from noise-corrupted source utterances.