Unsupervised Spoken Term Discovery on Untranscribed Speech

TL;DR

This paper explores unsupervised spoken term discovery in untranscribed speech, using neural networks and pattern analysis to identify keywords and topics without prior linguistic knowledge.

Contribution

It introduces a novel unsupervised system combining acoustic segment modeling and pattern discovery, utilizing neural network features for zero-resource language processing.

Findings

Bottleneck features improve acoustic segment modeling.

System successfully discovers keywords aligning with transcriptions.

Keywords enable effective topic comparison using IR techniques.

Abstract

(Part of the abstract) In this thesis, we investigate the use of unsupervised spoken term discovery in tackling this problem. Unsupervised spoken term discovery aims to discover topic-related terminologies in a speech without knowing the phonetic properties of the language and content. It can be further divided into two parts: Acoustic segment modelling (ASM) and unsupervised pattern discovery. ASM learns the phonetic structures of zero-resource language audio with no phonetic knowledge available, generating self-derived "phonemes". The audio are labelled with these "phonemes" to obtain "phoneme" sequences. Unsupervised pattern discovery searches for repetitive patterns in the "phoneme" sequences. The discovered patterns can be grouped to determine the keywords of the audio. Multilingual neural network with bottleneck layer is used for feature extraction. Experiments show that bottleneck features facilitate the training of ASM compared to conventional features such as MFCC. The unsupervised spoken term discovery system is experimented with online lectures covering different topics by different speakers. It is shown that the system learns the phonetic information of the language and can discover frequent spoken terms that align with text transcription. By using information retrieval technology such as word embedding and TFIDF, it is shown that the discovered keywords can be further used for topic comparison.