Neural Lexicon Reader: Reduce Pronunciation Errors in End-to-end TTS by Leveraging External Textual Knowledge

TL;DR

This paper introduces Neural Lexicon Reader, a framework that leverages external textual knowledge to improve pronunciation accuracy in end-to-end TTS systems, especially in low-resource scenarios.

Contribution

It proposes a novel Token2Knowledge attention module enabling end-to-end extraction of pronunciation knowledge from unstructured lexicon texts.

Findings

Significantly reduces pronunciation errors in low-resource Chinese TTS.

Demonstrates transferability of lexicon-reading capability to other languages.

Requires less data compared to traditional methods.

Abstract

End-to-end TTS requires a large amount of speech/text paired data to cover all necessary knowledge, particularly how to pronounce different words in diverse contexts, so that a neural model may learn such knowledge accordingly. But in real applications, such high demand of training data is hard to be satisfied and additional knowledge often needs to be injected manually. For example, to capture pronunciation knowledge on languages without regular orthography, a complicated grapheme-to-phoneme pipeline needs to be built based on a large structured pronunciation lexicon, leading to extra, sometimes high, costs to extend neural TTS to such languages. In this paper, we propose a framework to learn to automatically extract knowledge from unstructured external resources using a novel Token2Knowledge attention module. The framework is applied to build a TTS model named Neural Lexicon Reader that extracts pronunciations from raw lexicon texts in an end-to-end manner. Experiments show the proposed model significantly reduces pronunciation errors in low-resource, end-to-end Chinese TTS, and the lexicon-reading capability can be transferred to other languages with a smaller amount of data.