Towards Naturalistic Voice Conversion: NaturalVoices Dataset with an Automatic Processing Pipeline

TL;DR

This paper introduces NaturalVoices, a large-scale natural speech dataset for voice conversion, created through an automatic pipeline that extracts expressive, emotional speech from podcasts, addressing the scarcity of natural data in VC research.

Contribution

The study presents a novel, flexible pipeline for extracting natural, expressive speech data from podcasts, enabling the creation of a large-scale dataset for voice conversion.

Findings

The dataset contains over 3,800 hours of natural speech.

Objective and subjective evaluations confirm the dataset's effectiveness.

The pipeline successfully captures emotion and SNR from raw podcast data.

Abstract

Voice conversion (VC) research traditionally depends on scripted or acted speech, which lacks the natural spontaneity of real-life conversations. While natural speech data is limited for VC, our study focuses on filling in this gap. We introduce a novel data-sourcing pipeline that makes the release of a natural speech dataset for VC, named NaturalVoices. The pipeline extracts rich information in speech such as emotion and signal-to-noise ratio (SNR) from raw podcast data, utilizing recent deep learning methods and providing flexibility and ease of use. NaturalVoices marks a large-scale, spontaneous, expressive, and emotional speech dataset, comprising over 3,800 hours speech sourced from the original podcasts in the MSP-Podcast dataset. Objective and subjective evaluations demonstrate the effectiveness of using our pipeline for providing natural and expressive data for VC, suggesting the potential of NaturalVoices for broader speech generation tasks.