Marco-Voice Technical Report

TL;DR

Marco-Voice is a novel speech synthesis system that effectively disentangles speaker identity and emotion, enabling highly expressive, controllable, and natural speech generation with a new emotional speech dataset and state-of-the-art results.

Contribution

The paper introduces a speaker-emotion disentanglement mechanism and a new emotional speech dataset, advancing expressive neural speech synthesis.

Findings

Significant improvements in speech clarity and emotional richness.

Competitive performance in objective and subjective evaluations.

Effective manipulation of speaker and emotional styles independently.

Abstract

This paper presents a multifunctional speech synthesis system that integrates voice cloning and emotion control speech synthesis within a unified framework. The goal of this work is to address longstanding challenges in achieving highly expressive, controllable, and natural speech generation that faithfully preserves speaker identity across diverse linguistic and emotional contexts. Our approach introduces an effective speaker-emotion disentanglement mechanism with in-batch contrastive learning, enabling independent manipulation of speaker identity and eemotional style, as well as rotational emotional embedding integration method for smooth emotion control. To support comprehensive training and evaluation, we construct CSEMOTIONS, a high-quality emotional speech dataset containing 10 hours of Mandarin speech from six professional speakers across seven emotional categories. Extensive experiments demonstrate that our system, Marco-Voice, achieves substantial improvements in both objective and subjective metrics. Comprehensive evaluations and analysis were conducted, results show that MarcoVoice delivers competitive performance in terms of speech clarity and emotional richness, representing a substantial advance in the field of expressive neural speech synthesis. Our code and dataset are publicly available at https://github.com/AIDC-AI/Marco-Voice and https://huggingface.co/datasets/AIDC-AI/CSEMOTIONS respectively.