Personalized Keyword Spotting through Multi-task Learning

TL;DR

This paper introduces a multi-task learning approach for personalized keyword spotting, leveraging speaker verification to improve accuracy and reduce false alarms in user-specific scenarios.

Contribution

It proposes a novel multi-task learning framework combining keyword spotting and speaker verification for personalized KWS, with task-specific scoring functions for better adaptation.

Findings

Significant reduction in false alarm rates in personalized scenarios

Effective leveraging of speaker information improves KWS accuracy

Framework outperforms conventional methods in practical tests

Abstract

Keyword spotting (KWS) plays an essential role in enabling speech-based user interaction on smart devices, and conventional KWS (C-KWS) approaches have concentrated on detecting user-agnostic pre-defined keywords. However, in practice, most user interactions come from target users enrolled in the device which motivates to construct personalized keyword spotting. We design two personalized KWS tasks; (1) Target user Biased KWS (TB-KWS) and (2) Target user Only KWS (TO-KWS). To solve the tasks, we propose personalized keyword spotting through multi-task learning (PK-MTL) that consists of multi-task learning and task-adaptation. First, we introduce applying multi-task learning on keyword spotting and speaker verification to leverage user information to the keyword spotting system. Next, we design task-specific scoring functions to adapt to the personalized KWS tasks thoroughly. We evaluate our framework on conventional and personalized scenarios, and the results show that PK-MTL can dramatically reduce the false alarm rate, especially in various practical scenarios.