Mixed-EVC: Mixed Emotion Synthesis and Control in Voice Conversion

TL;DR

This paper introduces Mixed-EVC, a novel voice conversion framework that synthesizes and controls mixed emotional expressions in speech by leveraging discrete emotion labels and an attribute vector for nuanced emotional rendering.

Contribution

The paper presents a new EVC method that models mixed emotions and enhances control using an attribute vector and ranking-based SVM, surpassing traditional discrete emotion conversion methods.

Findings

Effective synthesis of mixed emotions confirmed by evaluations

Enhanced control over emotional expression demonstrated

Outperforms traditional discrete emotion conversion baselines

Abstract

Emotional voice conversion (EVC) traditionally targets the transformation of spoken utterances from one emotional state to another, with previous research mainly focusing on discrete emotion categories. This paper departs from the norm by introducing a novel perspective: a nuanced rendering of mixed emotions and enhancing control over emotional expression. To achieve this, we propose a novel EVC framework, Mixed-EVC, which only leverages discrete emotion training labels. We construct an attribute vector that encodes the relationships among these discrete emotions, which is predicted using a ranking-based support vector machine and then integrated into a sequence-to-sequence (seq2seq) EVC framework. Mixed-EVC not only learns to characterize the input emotional style but also quantifies its relevance to other emotions during training. As a result, users have the ability to assign these attributes to achieve their desired rendering of mixed emotions. Objective and subjective evaluations confirm the effectiveness of our approach in terms of mixed emotion synthesis and control while surpassing traditional baselines in the conversion of discrete emotions from one to another.