Temporal Dynamic Convolutional Neural Network for Text-Independent Speaker Verification and Phonemetic Analysis

TL;DR

This paper introduces a temporal dynamic CNN that adapts kernels over time to phoneme variations, improving speaker verification accuracy without explicit phoneme labels.

Contribution

The proposed TDY-CNN dynamically adapts kernels to phoneme variations over time, enhancing text-independent speaker verification performance.

Findings

Improved EER by 17.3% with TDY-ResNet-38(x0.5)

Adaptive kernels are phoneme-specific, especially in early layers

Temporal dynamic modeling enhances robustness in speaker verification

Abstract

In the field of text-independent speaker recognition, dynamic models that adapt along the time axis have been proposed to consider the phoneme-varying characteristics of speech. However, a detailed analysis of how dynamic models work depending on phonemes is insufficient. In this paper, we propose temporal dynamic CNN (TDY-CNN) that considers temporal variation of phonemes by applying kernels optimally adapting to each time bin. These kernels adapt to time bins by applying weighted sum of trained basis kernels. Then, an analysis of how adaptive kernels work on different phonemes in various layers is carried out. TDY-ResNet-38(x0.5) using six basis kernels improved an equal error rate (EER), the speaker verification performance, by 17.3% compared to the baseline model ResNet-38(x0.5). In addition, we showed that adaptive kernels depend on phoneme groups and are more phoneme-specific at early layers. The temporal dynamic model adapts itself to phonemes without explicitly given phoneme information during training, and results show the necessity to consider phoneme variation within utterances for more accurate and robust text-independent speaker verification.