FedMLAC: Mutual Learning Driven Heterogeneous Federated Audio Classification

TL;DR

FedMLAC is a novel federated learning framework for audio classification that effectively handles data and model heterogeneity and defends against data poisoning through mutual learning and a pruning aggregation strategy.

Contribution

It introduces a unified mutual learning approach with personalized models and a pruning-based aggregation to improve robustness and performance in heterogeneous federated audio classification.

Findings

Outperforms state-of-the-art methods in accuracy.

Demonstrates robustness against noisy and poisoned data.

Effective in diverse speech and non-speech tasks.

Abstract

Federated Learning (FL) offers a privacy-preserving framework for training audio classification (AC) models across decentralized clients without sharing raw data. However, Federated Audio Classification (FedAC) faces three major challenges: data heterogeneity, model heterogeneity, and data poisoning, which degrade performance in real-world settings. While existing methods often address these issues separately, a unified and robust solution remains underexplored. We propose FedMLAC, a mutual learning-based FL framework that tackles all three challenges simultaneously. Each client maintains a personalized local AC model and a lightweight, globally shared Plug-in model. These models interact via bidirectional knowledge distillation, enabling global knowledge sharing while adapting to local data distributions, thus addressing both data and model heterogeneity. To counter data poisoning, we introduce a Layer-wise Pruning Aggregation (LPA) strategy that filters anomalous Plug-in updates based on parameter deviations during aggregation. Extensive experiments on four diverse audio classification benchmarks, including both speech and non-speech tasks, show that FedMLAC consistently outperforms state-of-the-art baselines in classification accuracy and robustness to noisy data.