Phonetic-aware speaker embedding for far-field speaker verification

TL;DR

This paper introduces a joint-training framework that incorporates phonetic information into speaker embeddings to improve far-field speaker verification performance under noisy and reverberant conditions.

Contribution

The paper proposes a novel JTSS framework that aligns speaker embeddings with phonetic features, enhancing robustness in far-field speaker verification tasks.

Findings

Outperforms standard embeddings on VOiCES 2019 and VoxCeleb1 datasets.

Leveraging phonetic content improves robustness against noise and reverberation.

Demonstrates the effectiveness of phonetic-aware embeddings for far-field SV.

Abstract

When a speaker verification (SV) system operates far from the sound sourced, significant challenges arise due to the interference of noise and reverberation. Studies have shown that incorporating phonetic information into speaker embedding can improve the performance of text-independent SV. Inspired by this observation, we propose a joint-training speech recognition and speaker recognition (JTSS) framework to exploit phonetic content for far-field SV. The framework encourages speaker embeddings to preserve phonetic information by matching the frame-based feature maps of a speaker embedding network with wav2vec's vectors. The intuition is that phonetic information can preserve low-level acoustic dynamics with speaker information and thus partly compensate for the degradation due to noise and reverberation. Results show that the proposed framework outperforms the standard speaker embedding on the VOiCES Challenge 2019 evaluation set and the VoxCeleb1 test set. This indicates that leveraging phonetic information under far-field conditions is effective for learning robust speaker representations.