Does Single-channel Speech Enhancement Improve Keyword Spotting Accuracy? A Case Study

TL;DR

This study examines whether single-channel speech enhancement improves keyword spotting accuracy in noisy conditions, finding it effective only when the backend model is trained on clean speech, not noisy speech.

Contribution

It provides a comprehensive analysis of speech enhancement's impact on keyword spotting, including joint training and audio injection techniques, which are novel in this context.

Findings

SE improves KWS accuracy with clean-trained models in noisy environments

Joint training of SE and KWS models is explored but shows limited gains

Audio injection with optimized weighting can reduce distortions in enhanced speech

Abstract

Noise robustness is a key aspect of successful speech applications. Speech enhancement (SE) has been investigated to improve automatic speech recognition accuracy; however, its effectiveness for keyword spotting (KWS) is still under-investigated. In this paper, we conduct a comprehensive study on single-channel speech enhancement for keyword spotting on the Google Speech Command (GSC) dataset. To investigate robustness to noise, the GSC dataset is augmented with noise signals from the WSJ0 Hipster Ambient Mixtures (WHAM!) noise dataset. Our investigation includes not only applying SE before KWS but also performing joint training of the SE frontend and KWS backend models. Moreover, we explore audio injection, a common approach to reduce distortions by using a weighted average of the enhanced and original signals. Audio injection is then further optimized by using another model that predicts the weight for each utterance. Our investigation reveals that SE can improve KWS accuracy on noisy speech when the backend model is trained on clean speech; however, despite our extensive exploration, it is difficult to improve the KWS accuracy with SE when the backend is trained on noisy speech.