Selective Task offloading for Maximum Inference Accuracy and Energy efficient Real-Time IoT Sensing Systems

TL;DR

This paper presents a hybrid genetic algorithm for optimally allocating inference models in IoT edge systems, maximizing accuracy while respecting time and energy limits, and significantly improves speed and accuracy over existing methods.

Contribution

It introduces LGSTO, a lightweight hybrid genetic algorithm with novel termination and neighborhood techniques for efficient model allocation in resource-constrained IoT systems.

Findings

LGSTO is 3 times faster than comparable schemes.

LGSTO achieves higher average inference accuracy.

The problem is formulated as a strongly NP-hard multidimensional knapsack problem.

Abstract

The recent advancements in small-size inference models facilitated AI deployment on the edge. However, the limited resource nature of edge devices poses new challenges especially for real-time applications. Deploying multiple inference models (or a single tunable model) varying in size and therefore accuracy and power consumption, in addition to an edge server inference model, can offer a dynamic system in which the allocation of inference models to inference jobs is performed according to the current resource conditions. Therefore, in this work, we tackle the problem of selectively allocating inference models to jobs or offloading them to the edge server to maximize inference accuracy under time and energy constraints. This problem is shown to be an instance of the unbounded multidimensional knapsack problem which is considered a strongly NP-hard problem. We propose a lightweight hybrid genetic algorithm (LGSTO) to solve this problem. We introduce a termination condition and neighborhood exploration techniques for faster evolution of populations. We compare LGSTO with the Naive and Dynamic programming solutions. In addition to classic genetic algorithms using different reproduction methods including NSGA-II, and finally we compare to other evolutionary methods such as Particle swarm optimization (PSO) and Ant colony optimization (ACO). Experiment results show that LGSTO performed 3 times faster than the fastest comparable schemes while producing schedules with higher average accuracy.