Distilling On-Device Intelligence at the Network Edge

TL;DR

This paper explores how to efficiently and privately train machine learning models directly on devices at the network edge using fog computing, addressing communication, resource, and data challenges.

Contribution

It introduces fog ML frameworks that enable privacy-preserving, communication-efficient distributed on-device ML tailored for wireless edge environments.

Findings

Communication-efficient protocols improve model training speed.

Privacy-preserving methods protect user data during training.

Frameworks adapt to heterogeneous device resources.

Abstract

Devices at the edge of wireless networks are the last mile data sources for machine learning (ML). As opposed to traditional ready-made public datasets, these user-generated private datasets reflect the freshest local environments in real time. They are thus indispensable for enabling mission-critical intelligent systems, ranging from fog radio access networks (RANs) to driverless cars and e-Health wearables. This article focuses on how to distill high-quality on-device ML models using fog computing, from such user-generated private data dispersed across wirelessly connected devices. To this end, we introduce communication-efficient and privacy-preserving distributed ML frameworks, termed fog ML (FML), wherein on-device ML models are trained by exchanging model parameters, model outputs, and surrogate data. We then present advanced FML frameworks addressing wireless RAN characteristics, limited on-device resources, and imbalanced data distributions. Our study suggests that the full potential of FML can be reached by co-designing communication and distributed ML operations while accounting for heterogeneous hardware specifications, data characteristics, and user requirements.