Zero-Shot Federated Learning with New Classes for Audio Classification

TL;DR

This paper introduces a zero-shot federated learning framework for audio classification that effectively handles unseen classes and data heterogeneity, improving accuracy without accessing raw data.

Contribution

The paper proposes a novel zero-shot federated learning approach that synthesizes anonymized data impressions and manages heterogeneity in labels and models across users.

Findings

Achieved ~4.04% accuracy increase in keyword spotting.

Achieved ~4.26% accuracy increase in urban sound classification.

Effectively handles unseen classes and data heterogeneity.

Abstract

Federated learning is an effective way of extracting insights from different user devices while preserving the privacy of users. However, new classes with completely unseen data distributions can stream across any device in a federated learning setting, whose data cannot be accessed by the global server or other users. To this end, we propose a unified zero-shot framework to handle these aforementioned challenges during federated learning. We simulate two scenarios here -- 1) when the new class labels are not reported by the user, the traditional FL setting is used; 2) when new class labels are reported by the user, we synthesize Anonymized Data Impressions by calculating class similarity matrices corresponding to each device's new classes followed by unsupervised clustering to distinguish between new classes across different users. Moreover, our proposed framework can also handle statistical heterogeneities in both labels and models across the participating users. We empirically evaluate our framework on-device across different communication rounds (FL iterations) with new classes in both local and global updates, along with heterogeneous labels and models, on two widely used audio classification applications -- keyword spotting and urban sound classification, and observe an average deterministic accuracy increase of ~4.041% and ~4.258% respectively.