Early-exit deep neural networks for distorted images: providing an efficient edge offloading

TL;DR

This paper proposes expert side branches in early-exit DNNs trained on specific distortions, enabling better robustness and more efficient edge inference by accurately detecting distortions and reducing cloud offloading.

Contribution

Introducing distortion-specific expert branches in early-exit DNNs to enhance robustness and efficiency in edge inference for distorted images.

Findings

Improved edge inference accuracy with expert branches.

Reduced cloud offloading for distorted images.

Effective distortion detection on edge devices.

Abstract

Edge offloading for deep neural networks (DNNs) can be adaptive to the input's complexity by using early-exit DNNs. These DNNs have side branches throughout their architecture, allowing the inference to end earlier in the edge. The branches estimate the accuracy for a given input. If this estimated accuracy reaches a threshold, the inference ends on the edge. Otherwise, the edge offloads the inference to the cloud to process the remaining DNN layers. However, DNNs for image classification deals with distorted images, which negatively impact the branches' estimated accuracy. Consequently, the edge offloads more inferences to the cloud. This work introduces expert side branches trained on a particular distortion type to improve robustness against image distortion. The edge detects the distortion type and selects appropriate expert branches to perform the inference. This approach increases the estimated accuracy on the edge, improving the offloading decisions. We validate our proposal in a realistic scenario, in which the edge offloads DNN inference to Amazon EC2 instances.