A Glance is Enough: Extract Target Sentence By Looking at A keyword
Ying Shi, Dong Wang, Lantian Li, Jiqing Han

TL;DR
This paper presents a Transformer-based method to extract specific target sentences from multi-talker speech using only a keyword, significantly improving accuracy in noisy environments.
Contribution
It introduces a novel approach combining keyword embedding with cross-attention in Transformers for targeted speech extraction from overlapping speakers.
Findings
Achieves a PER of 26% on Librispeech at -3dB SNR
Outperforms baseline PER of 96%
Effective in noisy, multi-talker scenarios
Abstract
This paper investigates the possibility of extracting a target sentence from multi-talker speech using only a keyword as input. For example, in social security applications, the keyword might be "help", and the goal is to identify what the person who called for help is articulating while ignoring other speakers. To address this problem, we propose using the Transformer architecture to embed both the keyword and the speech utterance and then rely on the cross-attention mechanism to select the correct content from the concatenated or overlapping speech. Experimental results on Librispeech demonstrate that our proposed method can effectively extract target sentences from very noisy and mixed speech (SNR=-3dB), achieving a phone error rate (PER) of 26\%, compared to the baseline system's PER of 96%.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Text Analysis Techniques · Speech and dialogue systems · Natural Language Processing Techniques
MethodsMulti-Head Attention · Linear Layer · Position-Wise Feed-Forward Layer · Byte Pair Encoding · Attention Is All You Need · Dense Connections · Label Smoothing · Adam · Absolute Position Encodings · Residual Connection
