Re-ENACT: Reinforcement Learning for Emotional Speech Generation using Actor-Critic Strategy

TL;DR

This paper introduces Re-ENACT, a novel reinforcement learning approach that modifies speech prosody to alter perceived emotion, using an actor-critic strategy within a Bayesian framework, achieving results comparable to state-of-the-art models.

Contribution

It is the first method to modify prosodic features for emotion alteration in speech using actor-critic reinforcement learning with a Bayesian segmentation approach.

Findings

Successfully changes perceived emotion to target in speech

Performs on par with supervised and unsupervised state-of-the-art models

Provides a gradient computation solution for rhythm manipulation

Abstract

In this paper, we propose the first method to modify the prosodic features of a given speech signal using actor-critic reinforcement learning strategy. Our approach uses a Bayesian framework to identify contiguous segments of importance that links segments of the given utterances to perception of emotions in humans. We train a neural network to produce the variational posterior of a collection of Bernoulli random variables; our model applies a Markov prior on it to ensure continuity. A sample from this distribution is used for downstream emotion prediction. Further, we train the neural network to predict a soft assignment over emotion categories as the target variable. In the next step, we modify the prosodic features (pitch, intensity, and rhythm) of the masked segment to increase the score of target emotion. We employ an actor-critic reinforcement learning to train the prosody modifier by discretizing the space of modifications. Further, it provides a simple solution to the problem of gradient computation through WSOLA operation for rhythm manipulation. Our experiments demonstrate that this framework changes the perceived emotion of a given speech utterance to the target. Further, we show that our unified technique is on par with state-of-the-art emotion conversion models from supervised and unsupervised domains that require pairwise training.