Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation

TL;DR

This paper introduces a novel segmental contrastive predictive coding framework that models speech at the phoneme level, enabling unsupervised word segmentation and outperforming existing methods on standard datasets.

Contribution

The paper proposes a joint training framework with a differentiable boundary detector for segmental CPC, unifying phoneme and word segmentation tasks in an unsupervised manner.

Findings

Outperforms existing phoneme and word segmentation methods on TIMIT and Buckeye datasets.

Joint training of frame-level and segment-level encoders improves segmentation accuracy.

Boundary threshold and training schedule significantly impact segmentation quality.

Abstract

Automatic detection of phoneme or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ self-supervised training methods, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework that can model the signal structure at a higher level e.g. at the phoneme level. In this framework, a convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Typically, phoneme and word segmentation are treated as separate tasks. We unify them and experimentally show that our single model outperforms existing phoneme and word segmentation methods on TIMIT and Buckeye datasets. We analyze the impact of boundary threshold and when is the right time to include the segmental loss in the learning process.