S-vectors and TESA: Speaker Embeddings and a Speaker Authenticator Based on Transformer Encoder

TL;DR

This paper introduces s-vectors derived from a Transformer encoder for speaker embedding, demonstrating superior performance over x-vectors, and proposes TESA, a Transformer-based speaker verification architecture that outperforms traditional methods.

Contribution

The paper presents a novel speaker embedding method called s-vectors from Transformer encoders and introduces TESA, a Transformer-based speaker verification system, improving over existing approaches.

Findings

S-vectors outperform x-vectors in speaker recognition tasks.

TESA achieves better verification accuracy than PLDA-based methods.

Self-attention in Transformers effectively captures speaker characteristics.

Abstract

One of the most popular speaker embeddings is x-vectors, which are obtained from an architecture that gradually builds a larger temporal context with layers. In this paper, we propose to derive speaker embeddings from Transformer's encoder trained for speaker classification. Self-attention, on which Transformer's encoder is built, attends to all the features over the entire utterance and might be more suitable in capturing the speaker characteristics in an utterance. We refer to the speaker embeddings obtained from the proposed speaker classification model as s-vectors to emphasize that they are obtained from an architecture that heavily relies on self-attention. Through experiments, we demonstrate that s-vectors perform better than x-vectors. In addition to the s-vectors, we also propose a new architecture based on Transformer's encoder for speaker verification as a replacement for speaker verification based on conventional probabilistic linear discriminant analysis (PLDA). This architecture is inspired by the next sentence prediction task of bidirectional encoder representations from Transformers (BERT), and we feed the s-vectors of two utterances to verify whether they belong to the same speaker. We name this architecture the Transformer encoder speaker authenticator (TESA). Our experiments show that the performance of s-vectors with TESA is better than s-vectors with conventional PLDA-based speaker verification.