Scission: Performance-driven and Context-aware Cloud-Edge Distribution of Deep Neural Networks

TL;DR

Scission is an automated tool that optimizes the distribution of deep neural networks across devices, edge, and cloud resources by considering hardware capabilities, network conditions, and user constraints to maximize performance.

Contribution

The paper introduces Scission, a novel automated benchmarking tool that determines optimal DNN partitioning strategies considering multiple contextual factors.

Findings

Scission effectively identifies optimal DNN partitions for diverse hardware configurations.

Experimental results on 18 DNNs demonstrate performance improvements over manual partitioning.

Benchmarking overheads are manageable, enabling periodic re-optimization.

Abstract

Partitioning and distributing deep neural networks (DNNs) across end-devices, edge resources and the cloud has a potential twofold advantage: preserving privacy of the input data, and reducing the ingress bandwidth demand beyond the edge. However, for a given DNN, identifying the optimal partition configuration for distributing the DNN that maximizes performance is a significant challenge. This is because the combination of potential target hardware resources that maximizes performance and the sequence of layers of the DNN that should be distributed across the target resources needs to be determined, while accounting for user-defined objectives/constraints for partitioning. This paper presents Scission, a tool for automated benchmarking of DNNs on a given set of target device, edge and cloud resources for determining optimal partitions that maximize DNN performance. The decision-making approach is context-aware by capitalizing on hardware capabilities of the target resources, their locality, the characteristics of DNN layers, and the network condition. Experimental studies are carried out on 18 DNNs. The decisions made by Scission cannot be manually made by a human given the complexity and the number of dimensions affecting the search space. The benchmarking overheads of Scission allow for responding to operational changes periodically rather than in real-time. Scission is available for public download at https://github.com/qub-blesson/Scission.