TaigiSpeech: A Low-Resource Real-World Speech Intent Dataset and Preliminary Results with Scalable Data Mining In-the-Wild

TL;DR

TaigiSpeech introduces a real-world Taiwanese speech intent dataset for low-resource language applications, utilizing innovative data mining strategies to enable scalable dataset creation and preliminary intent detection results.

Contribution

The paper presents the first real-world Taiwanese speech intent dataset and explores novel data mining methods with minimal supervision for low-resource language data collection.

Findings

Successful collection of 3,000 utterances from older adults

Implementation of keyword match and multimodal data mining strategies

Preliminary intent detection results demonstrating feasibility

Abstract

Speech technologies have advanced rapidly and serve diverse populations worldwide. However, many languages remain underrepresented due to limited resources. In this paper, we introduce \textbf{TaigiSpeech}, a real-world speech intent dataset in Taiwanese Taigi (aka Taiwanese Hokkien/Southern Min), which is a low-resource and primarily spoken language. The dataset is collected from older adults, comprising 21 speakers with a total of 3k utterances. It is designed for practical intent detection scenarios, including healthcare and home assistant applications. To address the scarcity of labeled data, we explore two data mining strategies with two levels of supervision: keyword match data mining with LLM pseudo labeling via an intermediate language and an audio-visual framework that leverages multimodal cues with minimal textual supervision. This design enables scalable dataset construction for low-resource and unwritten spoken languages. TaigiSpeech will be released under the CC BY 4.0 license to facilitate broad adoption and research on low-resource and unwritten languages. The project website and the dataset can be found on https://kwchang.org/taigispeech.