Y-Vector: Multiscale Waveform Encoder for Speaker Embedding

TL;DR

This paper introduces a multi-scale waveform encoder for speaker verification that outperforms existing raw-waveform-based methods by leveraging convolutional branches at different time scales and advanced feature aggregation.

Contribution

The paper proposes a novel multi-scale waveform encoder with three convolution branches, squeeze-and-excitation blocks, and TDNN for improved speaker embedding from raw waveforms.

Findings

Outperforms existing raw-waveform-based speaker embeddings

Attends to different frequency bands at various scales

Produces a flatter overall frequency response

Abstract

State-of-the-art text-independent speaker verification systems typically use cepstral features or filter bank energies as speech features. Recent studies attempted to extract speaker embeddings directly from raw waveforms and have shown competitive results. In this paper, we propose a novel multi-scale waveform encoder that uses three convolution branches with different time scales to compute speech features from the waveform. These features are then processed by squeeze-and-excitation blocks, a multi-level feature aggregator, and a time delayed neural network (TDNN) to compute speaker embedding. We show that the proposed embeddings outperform existing raw-waveform-based speaker embeddings on speaker verification by a large margin. A further analysis of the learned filters shows that the multi-scale encoder attends to different frequency bands at its different scales while resulting in a more flat overall frequency response than any of the single-scale counterparts.