Voice Impersonation using Generative Adversarial Networks

TL;DR

This paper introduces a neural network framework based on GANs for voice impersonation, capable of convincingly mimicking target speaker qualities and styles across different genders, regardless of linguistic content.

Contribution

It presents a novel GAN-based speech style-transfer model that effectively captures voice quality and style for impersonation, addressing durational variability and cross-gender impersonation.

Findings

Generated speech samples are highly convincing.

Model successfully impersonates voices across genders.

Qualitative evaluations confirm effectiveness.

Abstract

Voice impersonation is not the same as voice transformation, although the latter is an essential element of it. In voice impersonation, the resultant voice must convincingly convey the impression of having been naturally produced by the target speaker, mimicking not only the pitch and other perceivable signal qualities, but also the style of the target speaker. In this paper, we propose a novel neural network based speech quality- and style- mimicry framework for the synthesis of impersonated voices. The framework is built upon a fast and accurate generative adversarial network model. Given spectrographic representations of source and target speakers' voices, the model learns to mimic the target speaker's voice quality and style, regardless of the linguistic content of either's voice, generating a synthetic spectrogram from which the time domain signal is reconstructed using the Griffin-Lim method. In effect, this model reframes the well-known problem of style-transfer for images as the problem of style-transfer for speech signals, while intrinsically addressing the problem of durational variability of speech sounds. Experiments demonstrate that the model can generate extremely convincing samples of impersonated speech. It is even able to impersonate voices across different genders effectively. Results are qualitatively evaluated using standard procedures for evaluating synthesized voices.