EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge

TL;DR

EdgeFM is a system that enhances open-set recognition on resource-limited edge devices by leveraging foundation models through edge-cloud cooperation, dynamic model switching, and selective data uploading, achieving high accuracy and low latency.

Contribution

This paper introduces EdgeFM, a novel system that effectively utilizes foundation models on edge devices via edge-cloud cooperation and dynamic adaptation, addressing resource constraints and open-set recognition.

Findings

Reduces end-to-end latency by up to 3.2x

Achieves 34.3% accuracy improvement over baseline

Demonstrates effectiveness on multiple datasets

Abstract

Deep Learning (DL) models have been widely deployed on IoT devices with the help of advancements in DL algorithms and chips. However, the limited resources of edge devices make these on-device DL models hard to be generalizable to diverse environments and tasks. Although the recently emerged foundation models (FMs) show impressive generalization power, how to effectively leverage the rich knowledge of FMs on resource-limited edge devices is still not explored. In this paper, we propose EdgeFM, a novel edge-cloud cooperative system with open-set recognition capability. EdgeFM selectively uploads unlabeled data to query the FM on the cloud and customizes the specific knowledge and architectures for edge models. Meanwhile, EdgeFM conducts dynamic model switching at run-time taking into account both data uncertainty and dynamic network variations, which ensures the accuracy always close to the original FM. We implement EdgeFM using two FMs on two edge platforms. We evaluate EdgeFM on three public datasets and two self-collected datasets. Results show that EdgeFM can reduce the end-to-end latency up to 3.2x and achieve 34.3% accuracy increase compared with the baseline.