Split Federated Learning Over Heterogeneous Edge Devices: Algorithm and Optimization

TL;DR

This paper introduces HSFL, a novel split federated learning framework that enables parallel training on heterogeneous edge devices, optimizing resources to reduce latency and improve convergence in resource-constrained environments.

Contribution

The paper proposes a new HSFL framework with a resource allocation algorithm that jointly optimizes computational and transmission resources for heterogeneous devices.

Findings

HSFL achieves faster convergence and higher accuracy on non-iid data.

The optimization algorithm significantly reduces training latency.

HSFL outperforms existing methods in heterogeneous settings.

Abstract

Split Learning (SL) is a promising collaborative machine learning approach, enabling resource-constrained devices to train models without sharing raw data, while reducing computational load and preserving privacy simultaneously. However, current SL algorithms face limitations in training efficiency and suffer from prolonged latency, particularly in sequential settings, where the slowest device can bottleneck the entire process due to heterogeneous resources and frequent data exchanges between clients and servers. To address these challenges, we propose the Heterogeneous Split Federated Learning (HSFL) framework, which allows resource-constrained clients to train their personalized client-side models in parallel, utilizing different cut layers. Aiming to mitigate the impact of heterogeneous environments and accelerate the training process, we formulate a latency minimization problem that optimizes computational and transmission resources jointly. Additionally, we design a resource allocation algorithm that combines the Sample Average Approximation (SAA), Genetic Algorithm (GA), Lagrangian relaxation and Branch and Bound (B\&B) methods to efficiently solve this problem. Simulation results demonstrate that HSFL outperforms other frameworks in terms of both convergence rate and model accuracy on heterogeneous devices with non-iid data, while the optimization algorithm is better than other baseline methods in reducing latency.