Golden Gemini is All You Need: Finding the Sweet Spots for Speaker Verification

TL;DR

This paper systematically explores stride configurations in ResNet models for speaker verification, identifying optimal settings called Golden Gemini that significantly improve performance and efficiency across multiple datasets.

Contribution

It introduces the Golden Gemini principle for designing 2D ResNet models tailored to speaker verification, leading to state-of-the-art results and a new benchmark.

Findings

Golden Gemini points improve EER and minDCF significantly

State-of-the-art ResNet baseline gains performance and efficiency

Golden Gemini is effective across various architectures and training conditions

Abstract

Previous studies demonstrate the impressive performance of residual neural networks (ResNet) in speaker verification. The ResNet models treat the time and frequency dimensions equally. They follow the default stride configuration designed for image recognition, where the horizontal and vertical axes exhibit similarities. This approach ignores the fact that time and frequency are asymmetric in speech representation. In this paper, we address this issue and look for optimal stride configurations specifically tailored for speaker verification. We represent the stride space on a trellis diagram, and conduct a systematic study on the impact of temporal and frequency resolutions on the performance and further identify two optimal points, namely Golden Gemini, which serves as a guiding principle for designing 2D ResNet-based speaker verification models. By following the principle, a state-of-the-art ResNet baseline model gains a significant performance improvement on VoxCeleb, SITW, and CNCeleb datasets with 7.70%/11.76% average EER/minDCF reductions, respectively, across different network depths (ResNet18, 34, 50, and 101), while reducing the number of parameters by 16.5% and FLOPs by 4.1%. We refer to it as Gemini ResNet. Further investigation reveals the efficacy of the proposed Golden Gemini operating points across various training conditions and architectures. Furthermore, we present a new benchmark, namely the Gemini DF-ResNet, using a cutting-edge model.