Exploration of End-to-end Synthesisers forZero Resource Speech Challenge 2020

TL;DR

This paper explores end-to-end zero resource speech synthesis by proposing four systems that improve synthesis quality through novel unit discovery and spectrogram mapping techniques, evaluated successfully in the Zerospeech 2020 challenge.

Contribution

It introduces four innovative zero resource TTS systems with modifications to spectrogram mapping, enhancing synthesis quality without transcriptions.

Findings

Achieved good synthesis quality in Zerospeech 2020 challenge

Demonstrated effectiveness of gender-specific modeling

Showed benefits of using x-vectors in mapping

Abstract

A Spoken dialogue system for an unseen language is referred to as Zero resource speech. It is especially beneficial for developing applications for languages that have low digital resources. Zero resource speech synthesis is the task of building text-to-speech (TTS) models in the absence of transcriptions. In this work, speech is modelled as a sequence of transient and steady-state acoustic units, and a unique set of acoustic units is discovered by iterative training. Using the acoustic unit sequence, TTS models are trained. The main goal of this work is to improve the synthesis quality of zero resource TTS system. Four different systems are proposed. All the systems consist of three stages: unit discovery, followed by unit sequence to spectrogram mapping, and finally spectrogram to speech inversion. Modifications are proposed to the spectrogram mapping stage. These modifications include training the mapping on voice data, using x-vectors to improve the mapping, two-stage learning, and gender-specific modelling. Evaluation of the proposed systems in the Zerospeech 2020 challenge shows that quite good quality synthesis can be achieved.