Accelerating Wireless Distributed Learning via Hybrid Split and Federated Learning Optimization

TL;DR

This paper proposes a hybrid split and federated learning framework to optimize distributed training in wireless networks, jointly tuning hyperparameters and resource allocation to accelerate convergence and improve accuracy.

Contribution

It introduces a novel joint optimization approach for mode selection, batch size, and resource allocation in hybrid FL and SL, with a two-stage solution for delay minimization.

Findings

Significant acceleration in convergence to target accuracy.

Effective joint optimization of hyperparameters and resources.

Improved training efficiency over existing methods.

Abstract

Federated learning (FL) and split learning (SL) are two effective distributed learning paradigms in wireless networks, enabling collaborative model training across mobile devices without sharing raw data. While FL supports low-latency parallel training, it may converge to less accurate model. In contrast, SL achieves higher accuracy through sequential training but suffers from increased delay. To leverage the advantages of both, hybrid split and federated learning (HSFL) allows some devices to operate in FL mode and others in SL mode. This paper aims to accelerate HSFL by addressing three key questions: 1) How does learning mode selection affect overall learning performance? 2) How does it interact with batch size? 3) How can these hyperparameters be jointly optimized alongside communication and computational resources to reduce overall learning delay? We first analyze convergence, revealing the interplay between learning mode and batch size. Next, we formulate a delay minimization problem and propose a two-stage solution: a block coordinate descent method for a relaxed problem to obtain a locally optimal solution, followed by a rounding algorithm to recover integer batch sizes with near-optimal performance. Experimental results demonstrate that our approach significantly accelerates convergence to the target accuracy compared to existing methods.