MOSS-TTSD: Text to Spoken Dialogue Generation

TL;DR

MOSS-TTSD is a novel spoken dialogue generation model that produces expressive, multi-party conversations with long-term coherence, multi-language support, and zero-shot voice cloning, addressing key challenges in dialogue synthesis.

Contribution

The paper introduces MOSS-TTSD, a new model for long-form, multi-party spoken dialogue generation with enhanced context modeling and a novel evaluation framework, TTSD-eval.

Findings

Outperforms existing models in dialogue synthesis quality.

Supports up to 60 minutes of synthesis and 5 speakers.

Effective zero-shot voice cloning from short reference clips.

Abstract

Spoken dialogue generation is crucial for applications like podcasts, dynamic commentary, and entertainment content, but poses significant challenges compared to single-utterance text-to-speech (TTS). Key requirements include accurate turn-taking, cross-turn acoustic consistency, and long-form stability, which current models often fail to address due to a lack of dialogue context modeling. To bridge this gap, we present MOSS-TTSD, a spoken dialogue synthesis model designed for expressive, multi-party conversational speech across multiple languages. With enhanced long-context modeling, MOSS-TTSD generates long-form spoken conversations from dialogue scripts with explicit speaker tags, supporting up to 60 minutes of single-pass synthesis, multi-party dialogue with up to 5 speakers, and zero-shot voice cloning from a short reference audio clip. The model supports various mainstream languages, including English and Chinese, and is adapted to several long-form scenarios. Additionally, to address limitations of existing evaluation methods, we propose TTSD-eval, an objective evaluation framework based on forced alignment that measures speaker attribution accuracy and speaker similarity without relying on speaker diarization tools. Both objective and subjective evaluation results show that MOSS-TTSD surpasses strong open-source and proprietary baselines in dialogue synthesis.