Edge Large AI Models: Collaborative Deployment and IoT Applications

TL;DR

This paper presents a framework for deploying large AI models at the network edge, enabling real-time IoT applications through collaborative training and modular inference to optimize resource use and reduce latency.

Contribution

It introduces a novel collaborative deployment framework for edge large AI models, including adaptive training and microservice-based inference, tailored for resource-constrained IoT environments.

Findings

Reduces communication and computation overhead during model fine-tuning.

Improves resource utilization and inference latency through modular architecture.

Enables diverse IoT applications with context-aware generative tasks.

Abstract

Large artificial intelligence models (LAMs) emulate human-like problem-solving capabilities across diverse domains, modalities, and tasks. By leveraging the communication and computation resources of geographically distributed edge devices, edge LAMs enable real-time intelligent services at the network edge. Unlike conventional edge AI, which relies on small or moderate-sized models for direct feature-to-prediction mappings, edge LAMs leverage the intricate coordination of modular components to enable context-aware generative tasks and multi-modal inference. We shall propose a collaborative deployment framework for edge LAM by characterizing the LAM intelligent capabilities and limited edge network resources. Specifically, we propose a collaborative training framework over heterogeneous edge networks that adaptively decomposes LAMs according to computation resources, data modalities, and training objectives, reducing communication and computation overheads during the fine-tuning process. Furthermore, we introduce a microservice-based inference framework that virtualizes the functional modules of edge LAMs according to their architectural characteristics, thereby improving resource utilization and reducing inference latency. The developed edge LAM will provide actionable solutions to enable diversified Internet-of-Things (IoT) applications, facilitated by constructing mappings from diverse sensor data to token representations and fine-tuning based on domain knowledge.