Resource-Aware Heterogeneous Federated Learning using Neural Architecture Search

TL;DR

This paper introduces RaFL, a resource-aware federated learning framework that uses neural architecture search to customize models for heterogeneous edge devices, improving efficiency and privacy in distributed AI training.

Contribution

The paper presents a novel resource-aware federated learning approach combining neural architecture search and multi-model fusion for heterogeneous edge devices.

Findings

RaFL achieves higher resource efficiency than state-of-the-art methods.

Customized models improve performance on non-IID data.

Multi-model fusion enhances aggregated learning results.

Abstract

Federated Learning (FL) is extensively used to train AI/ML models in distributed and privacy-preserving settings. Participant edge devices in FL systems typically contain non-independent and identically distributed (Non-IID) private data and unevenly distributed computational resources. Preserving user data privacy while optimizing AI/ML models in a heterogeneous federated network requires us to address data and system/resource heterogeneity. To address these challenges, we propose Resource-aware Federated Learning (RaFL). RaFL allocates resource-aware specialized models to edge devices using Neural Architecture Search (NAS) and allows heterogeneous model architecture deployment by knowledge extraction and fusion. Combining NAS and FL enables on-demand customized model deployment for resource-diverse edge devices. Furthermore, we propose a multi-model architecture fusion scheme allowing the aggregation of the distributed learning results. Results demonstrate RaFL's superior resource efficiency compared to SoTA.