Study on the temporal pooling used in deep neural networks for speaker verification

TL;DR

This paper investigates how different types of statistical pooling layers in deep neural networks affect speaker verification performance and other classification tasks, highlighting the importance of pooling content.

Contribution

It systematically evaluates various pooling strategies, including higher-order statistics, and demonstrates their impact on training dynamics and task accuracy.

Findings

Pooling content significantly influences speaker verification results.

Higher-order statistics like skewness and kurtosis improve performance.

External information can be revealed through pooling layer choices.

Abstract

The x-vector architecture has recently achieved state-of-the-art results on the speaker verification task. This architecture incorporates a central layer, referred to as temporal pooling, which stacks statistical parameters of the acoustic frame distribution. This work proposes to highlight the significant effect of the temporal pooling content on the training dynamics and task performance. An evaluation with different pooling layers is conducted, that is, including different statistical measures of central tendency. Notably, 3rd and 4th moment-based statistics (skewness and kurtosis) are also tested to complete the usual mean and standard-deviation parameters. Our experiments show the influence of the pooling layer content in terms of speaker verification performance, but also for several classification tasks (speaker, channel or text related), and allow to better reveal the presence of external information to the speaker identity depending on the layer content.