Split Knowledge Distillation for Large Models in IoT: Architecture, Challenges, and Solutions

TL;DR

This paper presents a split knowledge distillation framework that enables efficient deployment of large models in IoT systems by addressing privacy, resource constraints, and latency issues.

Contribution

It introduces a novel split knowledge distillation approach combining knowledge distillation and split learning tailored for IoT, with solutions for energy efficiency and low latency.

Findings

Framework reduces energy consumption in IoT model deployment

Enables training of smaller, accurate models on resource-limited devices

Case study demonstrates feasibility and performance improvements

Abstract

Large models (LMs) have immense potential in Internet of Things (IoT) systems, enabling applications such as intelligent voice assistants, predictive maintenance, and healthcare monitoring. However, training LMs on edge servers raises data privacy concerns, while deploying them directly on IoT devices is constrained by limited computational and memory resources. We analyze the key challenges of training LMs in IoT systems, including energy constraints, latency requirements, and device heterogeneity, and propose potential solutions such as dynamic resource management, adaptive model partitioning, and clustered collaborative training. Furthermore, we propose a split knowledge distillation framework to efficiently distill LMs into smaller, deployable versions for IoT devices while ensuring raw data remains local. This framework integrates knowledge distillation and split learning to minimize energy consumption and meet low model training delay requirements. A case study is presented to evaluate the feasibility and performance of the proposed framework.