Voice Filter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing module

TL;DR

This paper introduces Voice Filter, a novel low-resource TTS method that leverages voice conversion as a post-processing step, enabling high-quality speech synthesis with minimal target speaker data, outperforming existing few-shot techniques.

Contribution

It presents a new approach framing few-shot TTS as a voice conversion task, using a duration-controllable TTS system to create training data, and demonstrates superior performance with only one minute of speech.

Findings

Outperforms state-of-the-art few-shot TTS methods on objective metrics.

Achieves comparable quality to models trained on 30 times more data.

Effective with as little as one minute of target speaker speech.

Abstract

State-of-the-art text-to-speech (TTS) systems require several hours of recorded speech data to generate high-quality synthetic speech. When using reduced amounts of training data, standard TTS models suffer from speech quality and intelligibility degradations, making training low-resource TTS systems problematic. In this paper, we propose a novel extremely low-resource TTS method called Voice Filter that uses as little as one minute of speech from a target speaker. It uses voice conversion (VC) as a post-processing module appended to a pre-existing high-quality TTS system and marks a conceptual shift in the existing TTS paradigm, framing the few-shot TTS problem as a VC task. Furthermore, we propose to use a duration-controllable TTS system to create a parallel speech corpus to facilitate the VC task. Results show that the Voice Filter outperforms state-of-the-art few-shot speech synthesis techniques in terms of objective and subjective metrics on one minute of speech on a diverse set of voices, while being competitive against a TTS model built on 30 times more data.