Cross-speaker Emotion Transfer Based On Prosody Compensation for End-to-End Speech Synthesis

TL;DR

This paper introduces a prosody compensation module that enhances emotional information transfer in cross-speaker speech synthesis, ensuring emotional fidelity while preserving target speaker timbre.

Contribution

The paper proposes a novel prosody compensation module utilizing global context blocks to improve emotion transfer in end-to-end speech synthesis.

Findings

PCM effectively compensates emotional information loss.

The method maintains target speaker timbre.

Outperforms state-of-the-art models in emotion transfer.

Abstract

Cross-speaker emotion transfer speech synthesis aims to synthesize emotional speech for a target speaker by transferring the emotion from reference speech recorded by another (source) speaker. In this task, extracting speaker-independent emotion embedding from reference speech plays an important role. However, the emotional information conveyed by such emotion embedding tends to be weakened in the process to squeeze out the source speaker's timbre information. In response to this problem, a prosody compensation module (PCM) is proposed in this paper to compensate for the emotional information loss. Specifically, the PCM tries to obtain speaker-independent emotional information from the intermediate feature of a pre-trained ASR model. To this end, a prosody compensation encoder with global context (GC) blocks is introduced to obtain global emotional information from the ASR model's intermediate feature. Experiments demonstrate that the proposed PCM can effectively compensate the emotion embedding for the emotional information loss, and meanwhile maintain the timbre of the target speaker. Comparisons with state-of-the-art models show that our proposed method presents obvious superiority on the cross-speaker emotion transfer task.