A Neural TTS System with Parallel Prosody Transfer from Unseen Speakers

TL;DR

This paper introduces a neural TTS system capable of transferring prosody from unseen speakers' recordings to different TTS voices, enabling personalized speech synthesis without quality loss.

Contribution

The work presents the first neural TTS system with parallel prosody transfer from unseen speakers, enhancing expressiveness and personalization in speech synthesis.

Findings

Accurately transfers prosody from unseen speakers to TTS voices

Maintains speech quality and speaker identity

Subjective tests confirm naturalness and similarity

Abstract

Modern neural TTS systems are capable of generating natural and expressive speech when provided with sufficient amounts of training data. Such systems can be equipped with prosody-control functionality, allowing for more direct shaping of the speech output at inference time. In some TTS applications, it may be desirable to have an option that guides the TTS system with an ad-hoc speech recording exemplar to impose an implicit fine-grained, user-preferred prosodic realization for certain input prompts. In this work we present a first-of-its-kind neural TTS system equipped with such functionality to transfer the prosody from a parallel text recording from an unseen speaker. We demonstrate that the proposed system can precisely transfer the speech prosody from novel speakers to various trained TTS voices with no quality degradation, while preserving the target TTS speakers' identity, as evaluated by a set of subjective listening experiments.