EMPHASIS: An Emotional Phoneme-based Acoustic Model for Speech Synthesis System

TL;DR

EMPHASIS is a multi-lingual, phoneme-based acoustic model for speech synthesis that produces expressive, high-quality speech in real-time, incorporating emotional and prosodic features for Mandarin-English synthesis.

Contribution

It introduces a novel CBHG-based regression network with feature grouping for improved emotional speech synthesis in a multi-lingual framework.

Findings

Achieves better subjective quality than existing real-time systems

Capable of synthesizing expressive interrogative and exclamatory speech

Supports Mandarin-English bilingual speech synthesis

Abstract

We present EMPHASIS, an emotional phoneme-based acoustic model for speech synthesis system. EMPHASIS includes a phoneme duration prediction model and an acoustic parameter prediction model. It uses a CBHG-based regression network to model the dependencies between linguistic features and acoustic features. We modify the input and output layer structures of the network to improve the performance. For the linguistic features, we apply a feature grouping strategy to enhance emotional and prosodic features. The acoustic parameters are designed to be suitable for the regression task and waveform reconstruction. EMPHASIS can synthesize speech in real-time and generate expressive interrogative and exclamatory speech with high audio quality. EMPHASIS is designed to be a multi-lingual model and can synthesize Mandarin-English speech for now. In the experiment of emotional speech synthesis, it achieves better subjective results than other real-time speech synthesis systems.