Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation

TL;DR

This paper introduces a fully unsupervised, self-supervised contrastive learning model for phoneme boundary detection directly from raw speech waveforms, achieving state-of-the-art results across multiple datasets and languages.

Contribution

It presents a novel CNN-based model optimized with Noise-Contrastive Estimation for unsupervised phoneme segmentation, leveraging raw waveforms without manual annotations.

Findings

Outperforms existing unsupervised baselines on TIMIT and Buckeye datasets.

Achieves state-of-the-art performance in phoneme boundary detection.

Utilizes additional untranscribed data to improve cross-lingual generalization.

Abstract

We propose a self-supervised representation learning model for the task of unsupervised phoneme boundary detection. The model is a convolutional neural network that operates directly on the raw waveform. It is optimized to identify spectral changes in the signal using the Noise-Contrastive Estimation principle. At test time, a peak detection algorithm is applied over the model outputs to produce the final boundaries. As such, the proposed model is trained in a fully unsupervised manner with no manual annotations in the form of target boundaries nor phonetic transcriptions. We compare the proposed approach to several unsupervised baselines using both TIMIT and Buckeye corpora. Results suggest that our approach surpasses the baseline models and reaches state-of-the-art performance on both data sets. Furthermore, we experimented with expanding the training set with additional examples from the Librispeech corpus. We evaluated the resulting model on distributions and languages that were not seen during the training phase (English, Hebrew and German) and showed that utilizing additional untranscribed data is beneficial for model performance.