Decomposed Temporal Dynamic CNN: Efficient Time-Adaptive Network for Text-Independent Speaker Verification Explained with Speaker Activation Map

TL;DR

This paper introduces a decomposed temporal dynamic CNN that significantly reduces model size and improves accuracy in text-independent speaker verification by combining static and dynamic kernels, with detailed analysis of speaker information extraction.

Contribution

Proposes DTDY-CNNs using matrix decomposition to create efficient, adaptive kernels, outperforming previous models in speaker verification accuracy and size reduction.

Findings

Achieved 0.96% EER with DTDY-ResNet-34(x0.50)

Reduced model size by 64% compared to TDY-CNNs

Effectively extracts speaker info from formant and high-frequency unvoiced phonemes

Abstract

To extract accurate speaker information for text-independent speaker verification, temporal dynamic CNNs (TDY-CNNs) adapting kernels to each time bin was proposed. However, model size of TDY-CNN is too large and the adaptive kernel's degree of freedom is limited. To address these limitations, we propose decomposed temporal dynamic CNNs (DTDY-CNNs) which forms time-adaptive kernel by combining static kernel with dynamic residual based on matrix decomposition. Proposed DTDY-ResNet-34(x0.50) using attentive statistical pooling without data augmentation shows EER of 0.96%, which is better than other state-of-the-art methods. DTDY-CNNs are successful upgrade of TDY-CNNs, reducing the model size by 64% and enhancing the performance. We showed that DTDY-CNNs extract more accurate frame-level speaker embeddings as well compared to TDY-CNNs. Detailed behaviors of DTDY-ResNet-34(x0.50) on extraction of speaker information were analyzed using speaker activation map (SAM) produced by modified gradient-weighted class activation mapping (Grad-CAM) for speaker verification. DTDY-ResNet-34(x0.50) effectively extracts speaker information from not only formant frequencies but also high frequency information of unvoiced phonemes, thus explaining its outstanding performance on text-independent speaker verification.