ZSDEVC: Zero-Shot Diffusion-based Emotional Voice Conversion with Disentangled Mechanism

TL;DR

This paper presents ZSDEVC, a zero-shot diffusion-based emotional voice conversion method that effectively converts emotions in speech for unseen speakers, achieving high emotional accuracy and naturalness.

Contribution

It introduces a novel diffusion framework with disentangled mechanisms and expressive guidance for zero-shot emotional voice conversion, trained on large emotional speech datasets.

Findings

High emotional accuracy in converted speech

Enhanced naturalness and speech quality

Effective zero-shot conversion for unseen speakers

Abstract

The human voice conveys not just words but also emotional states and individuality. Emotional voice conversion (EVC) modifies emotional expressions while preserving linguistic content and speaker identity, improving applications like human-machine interaction. While deep learning has advanced EVC models for specific target speakers on well-crafted emotional datasets, existing methods often face issues with emotion accuracy and speech distortion. In addition, the zero-shot scenario, in which emotion conversion is applied to unseen speakers, remains underexplored. This work introduces a novel diffusion framework with disentangled mechanisms and expressive guidance, trained on a large emotional speech dataset and evaluated on unseen speakers across in-domain and out-of-domain datasets. Experimental results show that our method produces expressive speech with high emotional accuracy, naturalness, and quality, showcasing its potential for broader EVC applications.