HE2C: A Holistic Approach for Allocating Latency-Sensitive AI Tasks across Edge-Cloud

TL;DR

HE2C is a comprehensive framework that optimizes latency, energy, accuracy, and throughput for deep learning tasks across edge and cloud resources, improving performance and battery life in resource-constrained environments.

Contribution

HE2C introduces a holistic approach with modules for feasibility checking, resource allocation, and approximate computing to optimize multiple metrics simultaneously.

Findings

Significantly increases task throughput within deadlines.

Preserves battery life and accuracy on edge devices.

Reduces latency impact through approximate computing.

Abstract

The high computational, memory, and energy demands of Deep Learning (DL) applications often exceed the capabilities of battery-powered edge devices, creating difficulties in meeting task deadlines and accuracy requirements. Unlike previous solutions that optimize a single metric (e.g., accuracy or energy efficiency), HE2C framework is designed to holistically address the latency, memory, accuracy, throughput, and energy demands of DL applications across edge-cloud continuum, thereby, delivering a more comprehensive and effective user experience. HE2C comprises three key modules: (a) a "feasibility-check module that evaluates the likelihood of meeting deadlines across both edge and cloud resources; (b) a "resource allocation strategy" that maximizes energy efficiency without sacrificing the inference accuracy; and (c) a "rescue module" that enhances throughput by leveraging approximate computing to trade accuracy for latency when necessary. Our primary objective is to maximize system prolong battery lifespan, throughput, and accuracy while adhering to strict latency constraints. Experimental evaluations in the context of wearable technologies for blind and visually impaired users demonstrate that HE2C significantly improves task throughput via completing a larger number of tasks within their specified deadlines, while preserving edge device battery and maintaining prediction accuracy with minimal latency impact. These results underscore HE2C's potential as a robust solution for resource management in latency-sensitive, energy-constrained edge-to-cloud environments.