Federated Self-Supervised Learning for Acoustic Event Classification

TL;DR

This paper explores federated self-supervised learning for acoustic event classification, enabling privacy-preserving on-device representation learning that improves classifier performance without sharing raw data.

Contribution

It introduces a novel federated self-supervised learning approach for AEC that does not rely on pseudo labels or user-generated targets, enhancing privacy and performance.

Findings

Up to 20.3% relative precision improvement

Maintains recall while improving precision

Demonstrates feasibility of privacy-preserving on-device learning

Abstract

Standard acoustic event classification (AEC) solutions require large-scale collection of data from client devices for model optimization. Federated learning (FL) is a compelling framework that decouples data collection and model training to enhance customer privacy. In this work, we investigate the feasibility of applying FL to improve AEC performance while no customer data can be directly uploaded to the server. We assume no pseudo labels can be inferred from on-device user inputs, aligning with the typical use cases of AEC. We adapt self-supervised learning to the FL framework for on-device continual learning of representations, and it results in improved performance of the downstream AEC classifiers without labeled/pseudo-labeled data available. Compared to the baseline w/o FL, the proposed method improves precision up to 20.3\% relatively while maintaining the recall. Our work differs from prior work in FL in that our approach does not require user-generated learning targets, and the data we use is collected from our Beta program and is de-identified, to maximally simulate the production settings.