An Overview & Analysis of Sequence-to-Sequence Emotional Voice Conversion

TL;DR

This paper reviews recent sequence-to-sequence models for emotional voice conversion, highlighting their approaches, datasets, and challenges to guide future research in this evolving field.

Contribution

It systematically analyzes recent sequence-to-sequence EVC papers across multiple aspects, providing a comprehensive overview of the current state-of-the-art.

Findings

Sequence-to-sequence models effectively handle variable-length speech conversion.

Current models face challenges in capturing emotional nuances and speech rhythm.

The review identifies key research gaps and future directions in EVC.

Abstract

Emotional voice conversion (EVC) focuses on converting a speech utterance from a source to a target emotion; it can thus be a key enabling technology for human-computer interaction applications and beyond. However, EVC remains an unsolved research problem with several challenges. In particular, as speech rate and rhythm are two key factors of emotional conversion, models have to generate output sequences of differing length. Sequence-to-sequence modelling is recently emerging as a competitive paradigm for models that can overcome those challenges. In an attempt to stimulate further research in this promising new direction, recent sequence-to-sequence EVC papers were systematically investigated and reviewed from six perspectives: their motivation, training strategies, model architectures, datasets, model inputs, and evaluation methods. This information is organised to provide the research community with an easily digestible overview of the current state-of-the-art. Finally, we discuss existing challenges of sequence-to-sequence EVC.