Hetero-SplitEE: Split Learning of Neural Networks with Early Exits for Heterogeneous IoT Devices

TL;DR

Hetero-SplitEE introduces a split learning framework with early exits tailored for heterogeneous IoT devices, enabling efficient collaborative training of neural networks with device-specific split points.

Contribution

The paper proposes a novel split learning method that allows heterogeneous IoT devices to train shared neural networks with device-specific split points and introduces two cooperative training strategies.

Findings

Maintains competitive accuracy across datasets.

Supports diverse computational constraints effectively.

Enables practical deployment in heterogeneous IoT environments.

Abstract

The continuous scaling of deep neural networks has fundamentally transformed machine learning, with larger models demonstrating improved performance across diverse tasks. This growth in model size has dramatically increased the computational resources required for the training process. Consequently, distributed approaches, such as Federated Learning and Split Learning, have become essential paradigms for scalable deployment. However, existing Split Learning approaches assume client homogeneity and uniform split points across all participants. This critically limits their applicability to real-world IoT systems where devices exhibit heterogeneity in computational resources. To address this limitation, this paper proposes Hetero-SplitEE, a novel method that enables heterogeneous IoT devices to train a shared deep neural network in parallel collaboratively. By integrating heterogeneous early exits into hierarchical training, our approach allows each client to select distinct split points (cut layers) tailored to its computational capacity. In addition, we propose two cooperative training strategies, the Sequential strategy and the Averaging strategy, to facilitate this collaboration among clients with different split points. The Sequential strategy trains clients sequentially with a shared server model to reduce computational overhead. The Averaging strategy enables parallel client training with periodic cross-layer aggregation. Extensive experiments on CIFAR-10, CIFAR-100, and STL-10 datasets using ResNet-18 demonstrate that our method maintains competitive accuracy while efficiently supporting diverse computational constraints, enabling practical deployment of collaborative deep learning in heterogeneous IoT ecosystems.