ImKWS: Test-Time Adaptation for Keyword Spotting with Class Imbalance

TL;DR

ImKWS introduces a novel test-time adaptation method for keyword spotting that effectively handles class imbalance and environmental noise, improving accuracy without requiring labeled data during deployment.

Contribution

The paper proposes ImKWS, a new TTA approach that separates entropy minimization into reward and penalty branches and enforces consistency across transformations to address class imbalance.

Findings

ImKWS outperforms existing methods on Google Speech Commands dataset.

The approach maintains stable and reliable adaptation in imbalanced scenarios.

Code is publicly available for reproducibility.

Abstract

Keyword spotting (KWS) identifies words for voice assistants, but environmental noise frequently reduces accuracy. Standard adaptation fixes this issue and strictly requires original or labeled audio. Test time adaptation (TTA) solves this data constraint using only unlabeled test audio. However, current methods fail to handle the severe imbalance between rare keywords and frequent background sounds. Consequently, standard entropy minimization (EM) becomes overconfident and heavily biased toward the frequent background class. To overcome this problem, we propose a TTA method named ImKWS. Our approach splits the entropy process into a reward branch and a penalty branch with separate update strengths. Furthermore, we enforce consistency across multiple audio transformations to ensure stable model updates. Experiments on the Google Speech Commands dataset indicate ImKWS achieves reliable adaptation in realistic imbalanced scenarios. The code is available on GitHub.