Poformer: A simple pooling transformer for speaker verification

TL;DR

PoFormer introduces a transformer-based pooling layer for speaker verification that captures temporal information more effectively, leading to significant improvements in verification accuracy over existing pooling methods.

Contribution

The paper proposes PoFormer, a novel transformer-based pooling structure with modifications for stability, enhancing speaker embedding quality in verification tasks.

Findings

At least 13.00% improvement in EER

At least 9.12% improvement in minDCF

Outperforms existing pooling systems on various datasets

Abstract

Most recent speaker verification systems are based on extracting speaker embeddings using a deep neural network. The pooling layer in the network aims to aggregate frame-level features extracted by the backbone. In this paper, we propose a new transformer based pooling structure called PoFormer to enhance the ability of the pooling layer to capture information along the whole time axis. Different from previous works that apply attention mechanism in a simple way or implement the multi-head mechanism in serial instead of in parallel, PoFormer follows the initial transformer structure with some minor modifications like a positional encoding generator, drop path and LayerScale to make the training procedure more stable and to prevent overfitting. Evaluated on various datasets, PoFormer outperforms the existing pooling system with at least a 13.00% improvement in EER and a 9.12% improvement in minDCF.