AIVD: Adaptive Edge-Cloud Collaboration for Accurate and Efficient Industrial Visual Detection

TL;DR

The paper introduces AIVD, a collaborative edge-cloud framework that enhances industrial visual detection accuracy and efficiency by combining lightweight edge detectors with cloud-based large language models, supported by a novel fine-tuning and scheduling strategy.

Contribution

It presents a unified approach for precise localization and semantic understanding in industrial settings, with innovative fine-tuning and resource-aware scheduling methods.

Findings

Reduces resource consumption significantly.

Improves classification accuracy and semantic consistency.

Achieves higher throughput and lower latency.

Abstract

Multimodal large language models (MLLMs) demonstrate exceptional capabilities in semantic understanding and visual reasoning, yet they still face challenges in precise object localization and resource-constrained edge-cloud deployment. To address this, this paper proposes the AIVD framework, which achieves unified precise localization and high-quality semantic generation through the collaboration between lightweight edge detectors and cloud-based MLLMs. To enhance the cloud MLLM's robustness against edge cropped-box noise and scenario variations, we design an efficient fine-tuning strategy with visual-semantic collaborative augmentation, significantly improving classification accuracy and semantic consistency. Furthermore, to maintain high throughput and low latency across heterogeneous edge devices and dynamic network conditions, we propose a heterogeneous resource-aware dynamic scheduling algorithm. Experimental results demonstrate that AIVD substantially reduces resource consumption while improving MLLM classification performance and semantic generation quality. The proposed scheduling strategy also achieves higher throughput and lower latency across diverse scenarios.