AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data

TL;DR

AutoPrep is an automated framework that preprocesses in-the-wild speech data to improve quality, generate labels, and transcriptions, facilitating better use of large-scale speech datasets for speech technology applications.

Contribution

The paper introduces AutoPrep, a novel automatic preprocessing framework for in-the-wild speech data, addressing noise, segmentation, and labeling challenges without manual annotation.

Findings

AutoPrep achieves comparable speech quality scores to open-source TTS datasets.

The framework enables high speaker similarity in TTS systems trained on preprocessed data.

AutoPrep effectively automates data cleaning and annotation for large-scale speech datasets.

Abstract

Recently, the utilization of extensive open-sourced text data has significantly advanced the performance of text-based large language models (LLMs). However, the use of in-the-wild large-scale speech data in the speech technology community remains constrained. One reason for this limitation is that a considerable amount of the publicly available speech data is compromised by background noise, speech overlapping, lack of speech segmentation information, missing speaker labels, and incomplete transcriptions, which can largely hinder their usefulness. On the other hand, human annotation of speech data is both time-consuming and costly. To address this issue, we introduce an automatic in-the-wild speech data preprocessing framework (AutoPrep) in this paper, which is designed to enhance speech quality, generate speaker labels, and produce transcriptions automatically. The proposed AutoPrep framework comprises six components: speech enhancement, speech segmentation, speaker clustering, target speech extraction, quality filtering and automatic speech recognition. Experiments conducted on the open-sourced WenetSpeech and our self-collected AutoPrepWild corpora demonstrate that the proposed AutoPrep framework can generate preprocessed data with similar DNSMOS and PDNSMOS scores compared to several open-sourced TTS datasets. The corresponding TTS system can achieve up to 0.68 in-domain speaker similarity.