Dark Experience for Incremental Keyword Spotting

TL;DR

This paper introduces DE-KWS, a continual learning method for keyword spotting that uses dark knowledge to prevent performance loss on previous tasks without increasing model size, suitable for edge devices.

Contribution

DE-KWS is a novel continual learning approach that combines rehearsal and distillation using dark knowledge, addressing catastrophic forgetting in keyword spotting.

Findings

DE-KWS outperforms existing baselines in accuracy on Google Speech Commands dataset.

It maintains performance on previous tasks without increasing model size.

The method is practical for resource-constrained edge devices.

Abstract

Spoken keyword spotting (KWS) is crucial for identifying keywords within audio inputs and is widely used in applications like Apple Siri and Google Home, particularly on edge devices. Current deep learning-based KWS systems, which are typically trained on a limited set of keywords, can suffer from performance degradation when encountering new domains, a challenge often addressed through few-shot fine-tuning. However, this adaptation frequently leads to catastrophic forgetting, where the model's performance on original data deteriorates. Progressive continual learning (CL) strategies have been proposed to overcome this, but they face limitations such as the need for task-ID information and increased storage, making them less practical for lightweight devices. To address these challenges, we introduce Dark Experience for Keyword Spotting (DE-KWS), a novel CL approach that leverages dark knowledge to distill past experiences throughout the training process. DE-KWS combines rehearsal and distillation, using both ground truth labels and logits stored in a memory buffer to maintain model performance across tasks. Evaluations on the Google Speech Command dataset show that DE-KWS outperforms existing CL baselines in average accuracy without increasing model size, offering an effective solution for resource-constrained edge devices. The scripts are available on GitHub for the future research.