SmartSplit: Latency-Energy-Memory Optimisation for CNN Splitting on Smartphone Environment

TL;DR

SmartSplit is a novel approach that optimizes CNN splitting between smartphones and cloud servers to reduce latency, energy, and memory usage, making AI applications more efficient on mobile devices.

Contribution

We introduce SmartSplit, a genetic algorithm-based method for multi-objective optimization of CNN workload distribution on smartphones and cloud servers.

Findings

SmartSplit reduces latency, energy, and memory consumption compared to existing methods.

Splitting CNNs is feasible and effective for resource-constrained smartphones.

Our approach outperforms state-of-the-art techniques in multi-objective optimization.

Abstract

Artificial Intelligence has now taken centre stage in the smartphone industry owing to the need of bringing all processing close to the user and addressing privacy concerns. Convolution Neural Networks (CNNs), which are used by several AI applications, are highly resource and computation intensive. Although new generation smartphones come with AI-enabled chips, minimal memory and energy utilisation is essential as many applications are run concurrently on a smartphone. In light of this, optimising the workload on the smartphone by offloading a part of the processing to a cloud server is an important direction of research. In this paper, we analyse the feasibility of splitting CNNs between smartphones and cloud server by formulating a multi-objective optimisation problem that optimises the end-to-end latency, memory utilisation, and energy consumption. We design SmartSplit, a Genetic Algorithm with decision analysis based approach to solve the optimisation problem. Our experiments run with multiple CNN models show that splitting a CNN between a smartphone and a cloud server is feasible. The proposed approach, SmartSplit fares better when compared to other state-of-the-art approaches.