Expressive Prompting: Improving Emotion Intensity and Speaker Consistency in Zero-Shot TTS

TL;DR

This paper introduces a two-stage prompt selection method for zero-shot speech synthesis that enhances emotional intensity and speaker consistency by evaluating prompts with prosodic, perceptual, and semantic metrics.

Contribution

The proposed static and dynamic prompt selection strategy improves expressive speech synthesis by ensuring stable speaker identity and appropriate emotional cues in zero-shot TTS.

Findings

Enhanced emotional expression in synthesized speech.

Improved speaker similarity and stability.

Effective prompt selection demonstrated through experiments.

Abstract

Recent advancements in speech synthesis have enabled large language model (LLM)-based systems to perform zero-shot generation with controllable content, timbre, speaker identity, and emotion through input prompts. As a result, these models heavily rely on prompt design to guide the generation process. However, existing prompt selection methods often fail to ensure that prompts contain sufficiently stable speaker identity cues and appropriate emotional intensity indicators, which are crucial for expressive speech synthesis. To address this challenge, we propose a two-stage prompt selection strategy specifically designed for expressive speech synthesis. In the static stage (before synthesis), we first evaluate prompt candidates using pitch-based prosodic features, perceptual audio quality, and text-emotion coherence scores evaluated by an LLM. We further assess the candidates under a specific TTS model by measuring character error rate, speaker similarity, and emotional similarity between the synthesized and prompt speech. In the dynamic stage (during synthesis), we use a textual similarity model to select the prompt that is most aligned with the current input text. Experimental results demonstrate that our strategy effectively selects prompt to synthesize speech with both high-intensity emotional expression and robust speaker identity, leading to more expressive and stable zero-shot TTS performance. Audio samples and codes will be available at https://whyrrrrun.github.io/ExpPro.github.io/.