Towards Tailored Models on Private AIoT Devices: Federated Direct Neural Architecture Search

TL;DR

This paper introduces federated neural architecture search methods that enable AIoT devices to automatically find resource-efficient, tailored neural networks without compromising privacy, effectively handling data and hardware heterogeneity.

Contribution

It proposes FDNAS and CFDNAS frameworks that facilitate hardware-aware, device-specific neural architecture search in federated settings with non-IID data.

Findings

Achieves state-of-the-art accuracy-efficiency trade-offs on non-IID datasets.

Effectively handles data heterogeneity across devices.

Adapts neural architectures to diverse hardware platforms.

Abstract

Neural networks often encounter various stringent resource constraints while deploying on edge devices. To tackle these problems with less human efforts, automated machine learning becomes popular in finding various neural architectures that fit diverse Artificial Intelligence of Things (AIoT) scenarios. Recently, to prevent the leakage of private information while enable automated machine intelligence, there is an emerging trend to integrate federated learning and neural architecture search (NAS). Although promising as it may seem, the coupling of difficulties from both tenets makes the algorithm development quite challenging. In particular, how to efficiently search the optimal neural architecture directly from massive non-independent and identically distributed (non-IID) data among AIoT devices in a federated manner is a hard nut to crack. In this paper, to tackle this challenge, by leveraging the advances in ProxylessNAS, we propose a Federated Direct Neural Architecture Search (FDNAS) framework that allows for hardware-friendly NAS from non- IID data across devices. To further adapt to both various data distributions and different types of devices with heterogeneous embedded hardware platforms, inspired by meta-learning, a Cluster Federated Direct Neural Architecture Search (CFDNAS) framework is proposed to achieve device-aware NAS, in the sense that each device can learn a tailored deep learning model for its particular data distribution and hardware constraint. Extensive experiments on non-IID datasets have shown the state-of-the-art accuracy-efficiency trade-offs achieved by the proposed solution in the presence of both data and device heterogeneity.