Neural voice cloning with a few low-quality samples

TL;DR

This paper investigates speech synthesis from limited, low-quality samples of a target speaker by extracting speaker embeddings, comparing adaptation and speaker-encoder methods on LibriTTS and VCTK datasets.

Contribution

It introduces a speaker embedding extraction approach for low-quality data and compares two speaker mimicking methods across different datasets.

Findings

Speaker embedding extraction enables voice cloning with limited data.

Adaptation and speaker-encoder methods show different performance impacts.

Variety in datasets affects clarity and similarity in voice cloning.

Abstract

In this paper, we explore the possibility of speech synthesis from low quality found data using only limited number of samples of target speaker. We try to extract only the speaker embedding from found data of target speaker unlike previous works which tries to train the entire text-to-speech system on found data. Also, the two speaker mimicking approaches which are adaptation and speaker-encoder-based are applied on newly released LibriTTS dataset and previously released VCTK corpus to examine the impact of speaker variety on clarity and target-speaker-similarity .