Applying Phonological Features in Multilingual Text-To-Speech

TL;DR

This paper explores the use of phonological features in multilingual text-to-speech systems to generate native, non-native, and code-switched speech in English and Mandarin, demonstrating feasibility and insights into second language acquisition.

Contribution

It introduces a mapping from ARPABET/pinyin to phonological features and demonstrates their application in multilingual TTS for native and non-native speech synthesis.

Findings

Phonological features can be used as input for multilingual TTS.

Generated speech shows accented qualities relevant to second language learning.

Feasibility of phonological features in TTS is confirmed, with room for performance improvements.

Abstract

This study investigates whether phonological features can be applied in text-to-speech systems to generate native and non-native speech in English and Mandarin. We present a mapping of ARPABET/pinyin to SAMPA/SAMPA-SC and then to phonological features. We tested whether this mapping could lead to the successful generation of native, non-native, and code-switched speech in the two languages. We ran two experiments, one with a small dataset and one with a larger dataset. The results proved that phonological features could be used as a feasible input system, although further investigation is needed to improve model performance. The accented output generated by the TTS models also helps with understanding human second language acquisition processes.