Cross-Lingual Query-by-Example Spoken Term Detection: A Transformer-Based Approach

TL;DR

This paper presents a transformer-based, language-agnostic query-by-example spoken term detection model that leverages image processing techniques, showing significant improvements over CNN baselines in multiple languages.

Contribution

Introduces a novel, language-agnostic QbE-STD model combining transformer architecture and image processing techniques, enabling effective cross-lingual spoken term detection.

Findings

Achieves 19-54% performance improvement over CNN baseline

Effectively counts query term repetitions within audio

Offers faster processing than DTW, with some accuracy trade-offs

Abstract

Query-by-example spoken term detection (QbE-STD) is typically constrained by transcribed data scarcity and language specificity. This paper introduces a novel, language-agnostic QbE-STD model leveraging image processing techniques and transformer architecture. By employing a pre-trained XLSR-53 network for feature extraction and a Hough transform for detection, our model effectively searches for user-defined spoken terms within any audio file. Experimental results across four languages demonstrate significant performance gains (19-54%) over a CNN-based baseline. While processing time is improved compared to DTW, accuracy remains inferior. Notably, our model offers the advantage of accurately counting query term repetitions within the target audio.