Voicy: Zero-Shot Non-Parallel Voice Conversion in Noisy Reverberant Environments

TL;DR

Voicy is a novel zero-shot voice conversion framework designed to operate effectively in noisy and reverberant environments, outperforming existing methods in naturalness and speaker similarity.

Contribution

It introduces a multi-encoder architecture inspired by de-noising auto-encoders for non-parallel zero-shot voice conversion in challenging acoustic conditions.

Findings

Voicy outperforms existing VC methods in noisy reverberant settings.

The framework achieves higher naturalness and speaker similarity.

Validated on a noisy reverberant LibriSpeech dataset.

Abstract

Voice Conversion (VC) is a technique that aims to transform the non-linguistic information of a source utterance to change the perceived identity of the speaker. While there is a rich literature on VC, most proposed methods are trained and evaluated on clean speech recordings. However, many acoustic environments are noisy and reverberant, severely restricting the applicability of popular VC methods to such scenarios. To address this limitation, we propose Voicy, a new VC framework particularly tailored for noisy speech. Our method, which is inspired by the de-noising auto-encoders framework, is comprised of four encoders (speaker, content, phonetic and acoustic-ASR) and one decoder. Importantly, Voicy is capable of performing non-parallel zero-shot VC, an important requirement for any VC system that needs to work on speakers not seen during training. We have validated our approach using a noisy reverberant version of the LibriSpeech dataset. Experimental results show that Voicy outperforms other tested VC techniques in terms of naturalness and target speaker similarity in noisy reverberant environments.