CDIO: Cross-Domain Inference Optimization with Resource Preference Prediction for Edge-Cloud Collaboration

TL;DR

CDIO is a framework that optimizes resource allocation for edge-cloud video inference tasks by predicting resource preferences, significantly reducing resource consumption and energy use while maintaining accuracy.

Contribution

This paper introduces a novel cross-domain inference optimization framework that predicts resource preferences and guides resource allocation for improved efficiency in edge-cloud systems.

Findings

Achieves 20%-40% reduction in computing and bandwidth consumption.

Reduces energy consumption by over 40%.

Effectively meets accuracy and delay requirements.

Abstract

Currently, massive video tasks are processed by edge-cloud collaboration. However, the diversity of task requirements and the dynamics of resources pose great challenges to efficient inference, resulting in many wasted resources. In this paper, we present CDIO, a cross-domain inference optimization framework designed for edge-cloud collaboration. For diverse input tasks, CDIO can predict resource preference types by analyzing spatial complexity and processing requirements of the task. Subsequently, a cross-domain collaborative optimization algorithm is employed to guide resource allocation in the edge-cloud system. By ensuring that each task is matched with the ideal servers, the edge-cloud system can achieve higher efficiency inference. The evaluation results on public datasets demonstrate that CDIO can effectively meet the accuracy and delay requirements for task processing. Compared to state-of-the-art edge-cloud solutions, CDIO achieves a computing and bandwidth consumption reduction of 20%-40%. And it can reduce energy consumption by more than 40%.