ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Search

TL;DR

This paper introduces ESAI, a framework that optimizes split AI deployment on IoT devices by using early exiting and neural architecture search, reducing data transmission and maintaining high accuracy.

Contribution

The paper proposes a novel framework combining early exiting and neural architecture search for efficient split AI on IoT devices, reducing communication costs.

Findings

Only 40% of data needs server transmission

Achieves 92% overall accuracy

Improves accuracy of both client and server models

Abstract

Recently, deep neural networks have been outperforming conventional machine learning algorithms in many computer vision-related tasks. However, it is not computationally acceptable to implement these models on mobile and IoT devices and the majority of devices are harnessing the cloud computing methodology in which outstanding deep learning models are responsible for analyzing the data on the server. This can bring the communication cost for the devices and make the whole system useless in those times where the communication is not available. In this paper, a new framework for deploying on IoT devices has been proposed which can take advantage of both the cloud and the on-device models by extracting the meta-information from each sample's classification result and evaluating the classification's performance for the necessity of sending the sample to the server. Experimental results show that only 40 percent of the test data should be sent to the server using this technique and the overall accuracy of the framework is 92 percent which improves the accuracy of both client and server models.