FedOAED: Federated On-Device Autoencoder Denoiser for Heterogeneous Data under Limited Client Availability

TL;DR

FedOAED introduces an on-device autoencoder denoiser in federated learning to effectively address data heterogeneity and limited client participation, leading to improved model performance on vision tasks.

Contribution

The paper presents FedOAED, a novel federated learning algorithm that uses an autoencoder denoiser to reduce client-drift and variance caused by data heterogeneity and partial client availability.

Findings

FedOAED outperforms existing methods on multiple vision datasets.

The autoencoder denoiser effectively mitigates client-drift.

Enhanced robustness under Non-IID data conditions.

Abstract

Over the last few decades, machine learning (ML) and deep learning (DL) solutions have demonstrated their potential across many applications by leveraging large amounts of high-quality data. However, strict data-sharing regulations such as the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA) have prevented many data-driven applications from being realised. Federated Learning (FL), in which raw data never leaves local devices, has shown promise in overcoming these limitations. Although FL has grown rapidly in recent years, it still struggles with heterogeneity, which produces gradient noise, client-drift, and increased variance from partial client participation. In this paper, we propose FedOAED, a novel federated learning algorithm designed to mitigate client-drift arising from multiple local training updates and the variance induced by partial client participation. FedOAED incorporates an on-device autoencoder denoiser on the client side to mitigate client-drift and variance resulting from heterogeneous data under limited client availability. Experiments on multiple vision datasets under Non-IID settings demonstrate that FedOAED consistently outperforms state-of-the-art baselines.