Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?

TL;DR

This study compares top-down and bottom-up approaches for unsupervised speech segmentation and clustering, finding that simple bottom-up methods can perform as well as more complex top-down systems, with faster processing.

Contribution

The paper provides a comparative analysis of top-down and bottom-up segmentation methods, highlighting the effectiveness of simple bottom-up strategies and identifying clustering as a key bottleneck.

Findings

Both approaches achieve state-of-the-art results on ZeroSpeech benchmarks.

The bottom-up method is nearly five times faster than the top-down approach.

Clustering remains the main limiting factor in unsupervised word discovery.

Abstract

We investigate the problem of segmenting unlabeled speech into word-like units and clustering these to create a lexicon. Prior work can be categorized into two frameworks. Bottom-up methods first determine boundaries and then cluster the fixed segmented words into a lexicon. In contrast, top-down methods incorporate information from the clustered words to inform boundary selection. However, it is unclear whether top-down information is necessary to improve segmentation. To explore this, we look at two similar approaches that differ in whether top-down clustering informs boundary selection. Our simple bottom-up strategy predicts word boundaries using the dissimilarity between adjacent self-supervised features, then clusters the resulting segments to construct a lexicon. Our top-down system is an updated version of the ES-KMeans dynamic programming method that iteratively uses K-means to update its boundaries. On the five-language ZeroSpeech benchmarks, both approaches achieve comparable state-of-the-art results, with the bottom-up system being nearly five times faster. Through detailed analyses, we show that the top-down influence of ES-KMeans can be beneficial (depending on factors like the candidate boundaries), but in many cases the simple bottom-up method performs just as well. For both methods, we show that the clustering step is a limiting factor. Therefore, we recommend that future work focus on improved clustering techniques and learning more discriminative word-like representations. Project code repository: https://github.com/s-malan/prom-seg-clus.