Collaborative Execution of Deep Neural Networks on Internet of Things Devices

TL;DR

This paper proposes a collaborative approach for executing deep neural networks on IoT devices by leveraging their combined computing power to perform real-time inference, addressing resource and privacy limitations of cloud-based solutions.

Contribution

It introduces a novel collaborative network method that enables IoT devices to jointly perform DNN inference, optimizing resource use and real-time processing capabilities.

Findings

Effective collaboration of IoT devices for DNN inference demonstrated

Improved real-time processing with balanced distributed pipeline

Validated on Raspberry Pi with multiple DNN models

Abstract

With recent advancements in deep neural networks (DNNs), we are able to solve traditionally challenging problems. Since DNNs are compute intensive, consumers, to deploy a service, need to rely on expensive and scarce compute resources in the cloud. This approach, in addition to its dependability on high-quality network infrastructure and data centers, raises new privacy concerns. These challenges may limit DNN-based applications, so many researchers have tried optimize DNNs for local and in-edge execution. However, inadequate power and computing resources of edge devices along with small number of requests limits current optimizations applicability, such as batch processing. In this paper, we propose an approach that utilizes aggregated existing computing power of Internet of Things (IoT) devices surrounding an environment by creating a collaborative network. In this approach, IoT devices cooperate to conduct single-batch inferencing in real time. While exploiting several new model-parallelism methods and their distribution characteristics, our approach enhances the collaborative network by creating a balanced and distributed processing pipeline. We have illustrated our work using many Raspberry Pis with studying DNN models such as AlexNet, VGG16, Xception, and C3D.