Dual-Model Distillation for Efficient Action Classification with Hybrid Edge-Cloud Solution

TL;DR

This paper introduces a hybrid edge-cloud framework with Dual-Model Distillation (DMD) for efficient action classification, reducing computational costs while maintaining high accuracy in resource-limited settings.

Contribution

It presents a novel unsupervised data generation method, DMD, to train a lightweight switcher model that intelligently offloads inference to a large cloud model when needed.

Findings

Reduced computational overhead compared to large models alone

Improved accuracy over standalone edge models

Demonstrated scalability in resource-constrained environments

Abstract

As Artificial Intelligence models, such as Large Video-Language models (VLMs), grow in size, their deployment in real-world applications becomes increasingly challenging due to hardware limitations and computational costs. To address this, we design a hybrid edge-cloud solution that leverages the efficiency of smaller models for local processing while deferring to larger, more accurate cloud-based models when necessary. Specifically, we propose a novel unsupervised data generation method, Dual-Model Distillation (DMD), to train a lightweight switcher model that can predict when the edge model's output is uncertain and selectively offload inference to the large model in the cloud. Experimental results on the action classification task show that our framework not only requires less computational overhead, but also improves accuracy compared to using a large model alone. Our framework provides a scalable and adaptable solution for action classification in resource-constrained environments, with potential applications beyond healthcare. Noteworthy, while DMD-generated data is used for optimizing performance and resource usage in our pipeline, we expect the concept of DMD to further support future research on knowledge alignment across multiple models.