Multilingual acoustic word embeddings for zero-resource languages

TL;DR

This paper presents a neural network-based multilingual acoustic word embedding approach that enhances zero-resource language speech applications, demonstrating improved keyword spotting and semantic search in real-world scenarios.

Contribution

It introduces a new neural network model for multilingual AWEs that outperforms existing models and explores the impact of language choices on zero-resource language tasks.

Findings

Outperforms existing AWE models on zero-resource languages

Demonstrates robustness in hate speech keyword spotting in Swahili broadcasts

Improves semantic query-by-example search with novel models

Abstract

This research addresses the challenge of developing speech applications for zero-resource languages that lack labelled data. It specifically uses acoustic word embedding (AWE) -- fixed-dimensional representations of variable-duration speech segments -- employing multilingual transfer, where labelled data from several well-resourced languages are used for pertaining. The study introduces a new neural network that outperforms existing AWE models on zero-resource languages. It explores the impact of the choice of well-resourced languages. AWEs are applied to a keyword-spotting system for hate speech detection in Swahili radio broadcasts, demonstrating robustness in real-world scenarios. Additionally, novel semantic AWE models improve semantic query-by-example search.