Avoid Overfitting User Specific Information in Federated Keyword Spotting

TL;DR

This paper introduces FedKWS-UI, a federated keyword spotting method that employs adversarial learning and adaptive training to prevent overfitting to user-specific data, enhancing privacy and robustness.

Contribution

It proposes novel strategies, including adversarial learning and adaptive local training, to reduce overfitting to user-specific information in federated keyword spotting models.

Findings

FedKWS-UI effectively captures user-invariant features.

Experimental results show improved generalization across users.

The approach enhances privacy by preventing overfitting to individual data.

Abstract

Keyword spotting (KWS) aims to discriminate a specific wake-up word from other signals precisely and efficiently for different users. Recent works utilize various deep networks to train KWS models with all users' speech data centralized without considering data privacy. Federated KWS (FedKWS) could serve as a solution without directly sharing users' data. However, the small amount of data, different user habits, and various accents could lead to fatal problems, e.g., overfitting or weight divergence. Hence, we propose several strategies to encourage the model not to overfit user-specific information in FedKWS. Specifically, we first propose an adversarial learning strategy, which updates the downloaded global model against an overfitted local model and explicitly encourages the global model to capture user-invariant information. Furthermore, we propose an adaptive local training strategy, letting clients with more training data and more uniform class distributions undertake more local update steps. Equivalently, this strategy could weaken the negative impacts of those users whose data is less qualified. Our proposed FedKWS-UI could explicitly and implicitly learn user-invariant information in FedKWS. Abundant experimental results on federated Google Speech Commands verify the effectiveness of FedKWS-UI.