Non-Contrastive Self-Supervised Learning of Utterance-Level Speech Representations

TL;DR

This paper introduces a non-contrastive self-supervised learning approach, adapted from computer vision, to learn utterance-level speech representations without labels, demonstrating superior performance in speaker verification and emotion recognition tasks.

Contribution

It applies the DINO method to speech data, eliminating the need for negative sampling and achieving state-of-the-art results in speech representation learning.

Findings

DINO embeddings achieved 4.38% EER in speaker verification, outperforming contrastive methods.

Iterative pseudo-labeling further reduced EER to 1.89%.

Embeddings showed strong performance across multiple speech tasks.

Abstract

Considering the abundance of unlabeled speech data and the high labeling costs, unsupervised learning methods can be essential for better system development. One of the most successful methods is contrastive self-supervised methods, which require negative sampling: sampling alternative samples to contrast with the current sample (anchor). However, it is hard to ensure if all the negative samples belong to classes different from the anchor class without labels. This paper applies a non-contrastive self-supervised learning method on an unlabeled speech corpus to learn utterance-level embeddings. We used DIstillation with NO labels (DINO), proposed in computer vision, and adapted it to the speech domain. Unlike the contrastive methods, DINO does not require negative sampling. These embeddings were evaluated on speaker verification and emotion recognition. In speaker verification, the unsupervised DINO embedding with cosine scoring provided 4.38% EER on the VoxCeleb1 test trial. This outperforms the best contrastive self-supervised method by 40% relative in EER. An iterative pseudo-labeling training pipeline, not requiring speaker labels, further improved the EER to 1.89%. In emotion recognition, the DINO embedding performed 60.87, 79.21, and 56.98% in micro-f1 score on IEMOCAP, Crema-D, and MSP-Podcast, respectively. The results imply the generality of the DINO embedding to different speech applications.