Self-Adaptive Soft Voice Activity Detection using Deep Neural Networks for Robust Speaker Verification

TL;DR

This paper introduces a self-adaptive soft voice activity detection method that integrates deep neural networks into speaker verification systems, improving robustness in real-world environments through domain adaptation techniques.

Contribution

It proposes a novel self-adaptive soft VAD approach combining soft feature selection with unsupervised domain adaptation for enhanced speaker verification.

Findings

Significant performance improvement in real-world environments.

Effective domain adaptation with speech posterior-based and joint learning schemes.

Enhanced robustness of speaker verification systems.

Abstract

Voice activity detection (VAD), which classifies frames as speech or non-speech, is an important module in many speech applications including speaker verification. In this paper, we propose a novel method, called self-adaptive soft VAD, to incorporate a deep neural network (DNN)-based VAD into a deep speaker embedding system. The proposed method is a combination of the following two approaches. The first approach is soft VAD, which performs a soft selection of frame-level features extracted from a speaker feature extractor. The frame-level features are weighted by their corresponding speech posteriors estimated from the DNN-based VAD, and then aggregated to generate a speaker embedding. The second approach is self-adaptive VAD, which fine-tunes the pre-trained VAD on the speaker verification data to reduce the domain mismatch. Here, we introduce two unsupervised domain adaptation (DA) schemes, namely speech posterior-based DA (SP-DA) and joint learning-based DA (JL-DA). Experiments on a Korean speech database demonstrate that the verification performance is improved significantly in real-world environments by using self-adaptive soft VAD.