ECCENTRIC: Edge-Cloud Collaboration Framework for Distributed Inference Using Knowledge Adaptation

TL;DR

Eccentric is a novel edge-cloud inference framework that adapts knowledge to balance computation, communication, and performance, reducing costs while maintaining high accuracy in classification and object detection tasks.

Contribution

The paper introduces a new framework called Eccentric that learns models with different trade-offs and adapts knowledge from edge to cloud to optimize inference efficiency.

Findings

Reduces computation and communication costs during inference.

Maintains high performance in classification and object detection.

Effective across multiple machine learning tasks.

Abstract

The massive growth in the utilization of edge AI has made the applications of machine learning models ubiquitous in different domains. Despite the computation and communication efficiency of these systems, due to limited computation resources on edge devices, relying on more computationally rich systems on the cloud side is inevitable in most cases. Cloud inference systems can achieve the best performance while the computation and communication cost is dramatically increasing by the expansion of a number of edge devices relying on these systems. Hence, there is a trade-off between the computation, communication, and performance of these systems. In this paper, we propose a novel framework, dubbed as Eccentric that learns models with different levels of trade-offs between these conflicting objectives. This framework, based on an adaptation of knowledge from the edge model to the cloud one, reduces the computation and communication costs of the system during inference while achieving the best performance possible. The Eccentric framework can be considered as a new form of compression method suited for edge-cloud inference systems to reduce both computation and communication costs. Empirical studies on classification and object detection tasks corroborate the efficacy of this framework.