Split-Et-Impera: A Framework for the Design of Distributed Deep Learning Applications

TL;DR

Split-Et-Impera is a framework that optimizes the placement of neural network components across distributed systems by analyzing interpretability, simulating communication costs, and matching application requirements.

Contribution

It introduces a novel approach to determine optimal DNN split points without trial-and-error, considering interpretability and communication costs.

Findings

Effective identification of split points improves distributed DNN performance.

Simulation-based evaluation accelerates the design process.

Framework aligns DNN deployment with application quality and latency needs.

Abstract

Many recent pattern recognition applications rely on complex distributed architectures in which sensing and computational nodes interact together through a communication network. Deep neural networks (DNNs) play an important role in this scenario, furnishing powerful decision mechanisms, at the price of a high computational effort. Consequently, powerful state-of-the-art DNNs are frequently split over various computational nodes, e.g., a first part stays on an embedded device and the rest on a server. Deciding where to split a DNN is a challenge in itself, making the design of deep learning applications even more complicated. Therefore, we propose Split-Et-Impera, a novel and practical framework that i) determines the set of the best-split points of a neural network based on deep network interpretability principles without performing a tedious try-and-test approach, ii) performs a communication-aware simulation for the rapid evaluation of different neural network rearrangements, and iii) suggests the best match between the quality of service requirements of the application and the performance in terms of accuracy and latency time.