Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability

TL;DR

This paper introduces i-ETTS, a novel reinforcement learning-based interactive training method for emotional text-to-speech synthesis that significantly improves the perceptual recognizability of intended emotions.

Contribution

It presents the first application of reinforcement learning in ETTS, enhancing emotion discriminability through interaction with a speech emotion recognition model.

Findings

i-ETTS outperforms state-of-the-art baselines in emotion accuracy

The iterative training strategy improves speech emotion style quality

Reinforcement learning effectively enhances perceptual emotion recognition

Abstract

Emotional text-to-speech synthesis (ETTS) has seen much progress in recent years. However, the generated voice is often not perceptually identifiable by its intended emotion category. To address this problem, we propose a new interactive training paradigm for ETTS, denoted as i-ETTS, which seeks to directly improve the emotion discriminability by interacting with a speech emotion recognition (SER) model. Moreover, we formulate an iterative training strategy with reinforcement learning to ensure the quality of i-ETTS optimization. Experimental results demonstrate that the proposed i-ETTS outperforms the state-of-the-art baselines by rendering speech with more accurate emotion style. To our best knowledge, this is the first study of reinforcement learning in emotional text-to-speech synthesis.