Speech to Speech Synthesis for Voice Impersonation

TL;DR

This paper introduces STSSN, a novel speech-to-speech synthesis model that effectively performs voice impersonation by combining speech recognition and synthesis techniques, demonstrating superior realism compared to GAN-based methods.

Contribution

The paper presents STSSN, a new model for speech-to-speech style transfer that integrates current state-of-the-art systems for improved voice impersonation.

Findings

STSSN generates more convincing voice impersonation than GAN-based models.

The model produces realistic audio samples despite some capacity limitations.

Benchmark results favor STSSN over comparable generative adversarial approaches.

Abstract

Numerous models have shown great success in the fields of speech recognition as well as speech synthesis, but models for speech to speech processing have not been heavily explored. We propose Speech to Speech Synthesis Network (STSSN), a model based on current state of the art systems that fuses the two disciplines in order to perform effective speech to speech style transfer for the purpose of voice impersonation. We show that our proposed model is quite powerful, and succeeds in generating realistic audio samples despite a number of drawbacks in its capacity. We benchmark our proposed model by comparing it with a generative adversarial model which accomplishes a similar task, and show that ours produces more convincing results.