Voice activity detection in the wild: A data-driven approach using teacher-student training

TL;DR

This paper introduces a data-driven teacher-student framework for voice activity detection that leverages weak labels and large-scale real-world data to improve performance in noisy environments.

Contribution

It presents a novel teacher-student training approach for VAD that requires only weak labels, enabling effective training on noisy, real-world datasets.

Findings

Significant improvements in noisy and real-world data scenarios.

Outperforms existing unsupervised and supervised VAD methods.

Effective utilization of large-scale, unconstrained audio datasets.

Abstract

Voice activity detection is an essential pre-processing component for speech-related tasks such as automatic speech recognition (ASR). Traditional supervised VAD systems obtain frame-level labels from an ASR pipeline by using, e.g., a Hidden Markov model. These ASR models are commonly trained on clean and fully transcribed data, limiting VAD systems to be trained on clean or synthetically noised datasets. Therefore, a major challenge for supervised VAD systems is their generalization towards noisy, real-world data. This work proposes a data-driven teacher-student approach for VAD, which utilizes vast and unconstrained audio data for training. Unlike previous approaches, only weak labels during teacher training are required, enabling the utilization of any real-world, potentially noisy dataset. Our approach firstly trains a teacher model on a source dataset (Audioset) using clip-level supervision. After training, the teacher provides frame-level guidance to a student model on an unlabeled, target dataset. A multitude of student models trained on mid- to large-sized datasets are investigated (Audioset, Voxceleb, NIST SRE). Our approach is then respectively evaluated on clean, artificially noised, and real-world data. We observe significant performance gains in artificially noised and real-world scenarios. Lastly, we compare our approach against other unsupervised and supervised VAD methods, demonstrating our method's superiority.