REVERB-FL: Server-Side Adversarial and Reserve-Enhanced Federated Learning for Robust Audio Classification

TL;DR

REVERB-FL is a server-side defense mechanism for federated learning in audio classification, using a small reserve set and adversarial training to improve robustness against poisoning and client heterogeneity.

Contribution

It introduces a lightweight, server-side approach combining reserve data and retraining to counteract poisoning without increasing client-side burden.

Findings

Faster convergence and lower steady-state error compared to baseline federated averaging.

Effectively mitigates model poisoning under various data poisoning scenarios.

Validated on open-source audio datasets with different non-IID data partitions.

Abstract

Federated learning (FL) enables a privacy-preserving training paradigm for audio classification but is highly sensitive to client heterogeneity and poisoning attacks, where adversarially compromised clients can bias the global model and hinder the performance of audio classifiers. To mitigate the effects of model poisoning for audio signal classification, we present REVERB-FL, a lightweight, server-side defense that couples a small reserve set (approximately 5%) with pre- and post-aggregation retraining and adversarial training. After each local training round, the server refines the global model on the reserve set with either clean or additional adversarially perturbed data, thereby counteracting non-IID drift and mitigating potential model poisoning without adding substantial client-side cost or altering the aggregation process. We theoretically demonstrate the feasibility of our framework, showing faster convergence and a reduced steady-state error relative to baseline federated averaging. We validate our framework on two open-source audio classification datasets with varying IID and Dirichlet non-IID partitions and demonstrate that REVERB-FL mitigates global model poisoning under multiple designs of local data poisoning.