Profiling AI Models: Towards Efficient Computation Offloading in Heterogeneous Edge AI Systems

TL;DR

This paper proposes a profiling approach for AI models to improve computation offloading efficiency in heterogeneous Edge AI systems, addressing resource limitations and system heterogeneity for better performance.

Contribution

It introduces a profiling framework capturing model and hardware data to predict resource use and optimize offloading in Edge AI environments.

Findings

Initial experiments with 3,000+ runs show promising prediction accuracy.

Profiling data can improve resource allocation decisions.

Framework addresses heterogeneity and resource constraints in Edge AI.

Abstract

The rapid growth of end-user AI applications, such as computer vision and generative AI, has led to immense data and processing demands often exceeding user devices' capabilities. Edge AI addresses this by offloading computation to the network edge, crucial for future services in 6G networks. However, it faces challenges such as limited resources during simultaneous offloads and the unrealistic assumption of homogeneous system architecture. To address these, we propose a research roadmap focused on profiling AI models, capturing data about model types, hyperparameters, and underlying hardware to predict resource utilisation and task completion time. Initial experiments with over 3,000 runs show promise in optimising resource allocation and enhancing Edge AI performance.