Detect, Attend and Extract: Keyword Guided Target Speaker Extraction

TL;DR

This paper introduces DAE-TSE, a keyword-guided target speaker extraction framework that uses partial transcriptions as cues, enabling speaker extraction without needing pre-enrolled speech, and demonstrating superior performance over traditional methods.

Contribution

It presents the first use of partial transcriptions as cues for target speaker extraction, providing a flexible alternative to enrollment-based systems.

Findings

DAE-TSE outperforms standard enrollment-based TSE systems.

First study to utilize partial transcription as a cue in TSE.

Code and demo are publicly available.

Abstract

Target speaker extraction (TSE) aims to extract the speech of a target speaker from mixtures containing multiple competing speakers. Conventional TSE systems predominantly rely on speaker cues, such as pre-enrolled speech, to identify and isolate the target speaker. However, in many practical scenarios, clean enrollment utterances are unavailable, limiting the applicability of existing approaches. In this work, we propose DAE-TSE, a keyword-guided TSE framework that specifies the target speaker through distinct keywords they utter. By leveraging keywords (i.e., partial transcriptions) as cues, our approach provides a flexible and practical alternative to enrollment-based TSE. DAE-TSE follows the Detect-Attend-Extract (DAE) paradigm: it first detects the presence of the given keywords, then attends to the corresponding speaker based on the keyword content, and finally extracts the target speech. Experimental results demonstrate that DAE-TSE outperforms standard TSE systems that rely on clean enrollment speech. To the best of our knowledge, this is the first study to utilize partial transcription as a cue for specifying the target speaker in TSE, offering a flexible and practical solution for real-world scenarios. Our code and demo page are now publicly available.