Enhancing Few-shot Keyword Spotting Performance through Pre-Trained Self-supervised Speech Models

TL;DR

This paper introduces a novel training scheme using self-supervised speech models and knowledge distillation to significantly improve few-shot keyword spotting accuracy on edge devices.

Contribution

It proposes a new training approach leveraging Wav2Vec 2.0 and attention-based dimensionality reduction for enhanced FS-KWS performance.

Findings

10-shot classification accuracy improved from 33.4% to 74.1% on GSC dataset

Enhanced inter-class separability and intra-class compactness with Sub-center ArcFace loss

Effective deployment on resource-constrained edge devices

Abstract

Keyword Spotting plays a critical role in enabling hands-free interaction for battery-powered edge devices. Few-Shot Keyword Spotting (FS-KWS) addresses the scalability and adaptability challenges of traditional systems by enabling recognition of custom keywords with only a few examples. However, existing FS-KWS systems achieve subpar accuracy at desirable false acceptance rates, particularly in resource-constrained edge environments. To address these issues, we propose a training scheme that leverages self-supervised learning models for robust feature extraction, dimensionality reduction, and knowledge distillation. The teacher model, based on Wav2Vec 2.0 is trained using Sub-center ArcFace loss, which enhances inter-class separability and intra-class compactness. To enable efficient deployment on edge devices, we introduce attention-based dimensionality reduction and train a standard lightweight ResNet15 student model. We evaluate the proposed approach on the English portion of the Multilingual Spoken Words Corpus (MSWC) and the Google Speech Commands (GSC) datasets. Notably, the proposed training method improves the 10-shot classification accuracy from 33.4% to 74.1% on 11 classes at 1% false alarm accuracy on the GSC dataset, thus making it significantly better-suited for a real use case scenario.