Personalized Hierarchical Split Federated Learning in Wireless Networks

TL;DR

This paper introduces a personalized hierarchical split federated learning algorithm designed for wireless networks, improving personalization and efficiency by training only parts of the model and fine-tuning classifiers for individual clients.

Contribution

The paper proposes a novel PHSFL algorithm that enhances personalization in split federated learning by training only the model body and fine-tuning classifiers, supported by theoretical analysis and empirical results.

Findings

Global model with untrained classifier performs comparably to existing methods.

Fine-tuned models significantly improve personalized performance.

Theoretical analysis clarifies the effects of model splitting and aggregation.

Abstract

Extreme resource constraints make large-scale machine learning (ML) with distributed clients challenging in wireless networks. On the one hand, large-scale ML requires massive information exchange between clients and server(s). On the other hand, these clients have limited battery and computation powers that are often dedicated to operational computations. Split federated learning (SFL) is emerging as a potential solution to mitigate these challenges, by splitting the ML model into client-side and server-side model blocks, where only the client-side block is trained on the client device. However, practical applications require personalized models that are suitable for the client's personal task. Motivated by this, we propose a personalized hierarchical split federated learning (PHSFL) algorithm that is specially designed to achieve better personalization performance. More specially, owing to the fact that regardless of the severity of the statistical data distributions across the clients, many of the features have similar attributes, we only train the body part of the federated learning (FL) model while keeping the (randomly initialized) classifier frozen during the training phase. We first perform extensive theoretical analysis to understand the impact of model splitting and hierarchical model aggregations on the global model. Once the global model is trained, we fine-tune each client classifier to obtain the personalized models. Our empirical findings suggest that while the globally trained model with the untrained classifier performs quite similarly to other existing solutions, the fine-tuned models show significantly improved personalized performance.