Distributed Deep Neural Networks over the Cloud, the Edge and End Devices

TL;DR

This paper introduces distributed deep neural networks (DDNNs) that operate across cloud, edge, and end devices, enabling scalable, fault-tolerant, and privacy-preserving inference with reduced communication costs.

Contribution

The paper presents a novel framework for mapping and jointly training DNN sections across a distributed hierarchy, improving efficiency and robustness in sensor fusion and inference.

Findings

DDNN reduces communication cost by over 20x compared to raw data offloading.

DDNN enables localized inference at edge and end devices.

The system improves object recognition accuracy by exploiting sensor geographical diversity.

Abstract

We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate inference of a deep neural network (DNN) in the cloud, a DDNN also allows fast and localized inference using shallow portions of the neural network at the edge and end devices. When supported by a scalable distributed computing hierarchy, a DDNN can scale up in neural network size and scale out in geographical span. Due to its distributed nature, DDNNs enhance sensor fusion, system fault tolerance and data privacy for DNN applications. In implementing a DDNN, we map sections of a DNN onto a distributed computing hierarchy. By jointly training these sections, we minimize communication and resource usage for devices and maximize usefulness of extracted features which are utilized in the cloud. The resulting system has built-in support for automatic sensor fusion and fault tolerance. As a proof of concept, we show a DDNN can exploit geographical diversity of sensors to improve object recognition accuracy and reduce communication cost. In our experiment, compared with the traditional method of offloading raw sensor data to be processed in the cloud, DDNN locally processes most sensor data on end devices while achieving high accuracy and is able to reduce the communication cost by a factor of over 20x.