DisfluencySpeech -- Single-Speaker Conversational Speech Dataset with Paralanguage

TL;DR

This paper introduces DisfluencySpeech, a high-quality English speech dataset with paralanguage, enabling better training of conversational TTS models that include non-lexical sounds like laughing and sighing.

Contribution

It provides a single-speaker, studio-quality dataset with annotated paralanguage and multiple transcripts, facilitating the development of TTS systems that generate expressive, disfluent speech.

Findings

Benchmark TTS models trained on different transcript levels.

Dataset includes nearly 10 hours of expressive speech.

Simulates realistic informal conversations.

Abstract

Laughing, sighing, stuttering, and other forms of paralanguage do not contribute any direct lexical meaning to speech, but they provide crucial propositional context that aids semantic and pragmatic processes such as irony. It is thus important for artificial social agents to both understand and be able to generate speech with semantically-important paralanguage. Most speech datasets do not include transcribed non-lexical speech sounds and disfluencies, while those that do are typically multi-speaker datasets where each speaker provides relatively little audio. This makes it challenging to train conversational Text-to-Speech (TTS) synthesis models that include such paralinguistic components. We thus present DisfluencySpeech, a studio-quality labeled English speech dataset with paralanguage. A single speaker recreates nearly 10 hours of expressive utterances from the Switchboard-1 Telephone Speech Corpus (Switchboard), simulating realistic informal conversations. To aid the development of a TTS model that is able to predictively synthesise paralanguage from text without such components, we provide three different transcripts at different levels of information removal (removal of non-speech events, removal of non-sentence elements, and removal of false starts), as well as benchmark TTS models trained on each of these levels.