Whilter: A Whisper-based Data Filter for "In-the-Wild" Speech Corpora Using Utterance-level Multi-Task Classification

TL;DR

Whilter is a multitask model leveraging Whisper encoder to filter undesirable samples in large in-the-wild speech datasets, improving data quality for speech recognition tasks.

Contribution

It introduces a novel multitask classification approach using Whisper for filtering diverse undesirable speech samples and provides an annotated dataset for in-the-wild corpora.

Findings

Achieves over 85% F1 scores on key classification tasks.

Reduces processing time compared to single-task methods.

Outperforms state-of-the-art classifiers on speech-specific classes.

Abstract

Large-scale in-the-wild speech datasets have become more prevalent in recent years due to increased interest in models that can learn useful features from unlabelled data for tasks such as speech recognition or synthesis. These datasets often contain undesirable features, such as multiple speakers, non-target languages, and music, which may impact model learning. The Whilter model is proposed as a multitask solution to identify these undesirable samples. Whilter uses a Whisper encoder with an attention-based classifier to solve five diverse classification problems at once. In addition, an annotated dataset is published for a subset of two popular in-the-wild corpora. Whilter achieves F1 scores above 85% and equal error rates of 6.5% to 7.8% for three of five subtasks, outperforming a state-of-the-art BEATs classifier on speech-specific classes, with a notable decrease in processing time compared to a combination of single-task alternatives.