Spoken-Term Discovery using Discrete Speech Units

TL;DR

DUSTED is a novel method for zero-resource speech processing that discovers spoken terms by encoding audio into discrete units and finding repeated patterns, achieving state-of-the-art results on the ZeroSpeech Challenge.

Contribution

This paper introduces DUSTED, a new approach using self-supervised discrete units and bioinformatics-inspired algorithms for improved spoken-term discovery.

Findings

Achieves state-of-the-art results on ZeroSpeech Challenge

Finds longer word- or phrase-like patterns

Improves pattern coverage and consistency across speakers

Abstract

Discovering a lexicon from unlabeled audio is a longstanding challenge for zero-resource speech processing. One approach is to search for frequently occurring patterns in speech. We revisit this idea with DUSTED: Discrete Unit Spoken-TErm Discovery. Leveraging self-supervised models, we encode input audio into sequences of discrete units. Next, we find repeated patterns by searching for similar unit sub-sequences, inspired by alignment algorithms from bioinformatics. Since discretization discards speaker information, DUSTED finds better matches across speakers, improving the coverage and consistency of the discovered patterns. We demonstrate these improvements on the ZeroSpeech Challenge, achieving state-of-the-art results on the spoken-term discovery track. Finally, we analyze the duration distribution of the patterns, showing that our method finds longer word- or phrase-like terms.