Shoggoth: Towards Efficient Edge-Cloud Collaborative Real-Time Video Inference via Adaptive Online Learning

TL;DR

Shoggoth is an innovative edge-cloud system that enhances real-time video inference accuracy and efficiency through adaptive online learning and dynamic resource management, addressing data drift and resource constraints.

Contribution

It introduces a novel adaptive online knowledge distillation framework for edge-cloud video inference, improving accuracy and reducing bandwidth and computational costs.

Findings

15%-20% accuracy improvement over edge-only methods

Fewer network costs than cloud-only approaches

Effective adaptation to changing video scenes

Abstract

This paper proposes Shoggoth, an efficient edge-cloud collaborative architecture, for boosting inference performance on real-time video of changing scenes. Shoggoth uses online knowledge distillation to improve the accuracy of models suffering from data drift and offloads the labeling process to the cloud, alleviating constrained resources of edge devices. At the edge, we design adaptive training using small batches to adapt models under limited computing power, and adaptive sampling of training frames for robustness and reducing bandwidth. The evaluations on the realistic dataset show 15%-20% model accuracy improvement compared to the edge-only strategy and fewer network costs than the cloud-only strategy.