Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning

TL;DR

This paper introduces a hybrid federated split learning framework that combines federated learning and split learning, utilizing a GAN-powered multi-objective optimization to balance training time and energy consumption in wireless networks.

Contribution

It proposes a novel HFSL framework with a parallel computing scheme and a GAN-based multi-objective optimization algorithm for resource allocation and model splitting.

Findings

The GAN-powered algorithm effectively finds Pareto optimal solutions.

HFSL outperforms traditional federated learning in experiments.

The scheme reduces computational idleness and improves convergence.

Abstract

As an edge intelligence algorithm for multi-device collaborative training, federated learning (FL) can reduce the communication burden but increase the computing load of wireless devices. In contrast, split learning (SL) can reduce the computing load of devices by using model splitting and assignment, but increase the communication burden to transmit intermediate results. In this paper, to exploit the advantages of FL and SL, we propose a hybrid federated split learning (HFSL) framework in wireless networks, which combines the multi-worker parallel update of FL and flexible splitting of SL. To reduce the computational idleness in model splitting, we design a parallel computing scheme for model splitting without label sharing, and theoretically analyze the influence of the delayed gradient caused by the scheme on the convergence speed. Aiming to obtain the trade-off between the training time and energy consumption, we optimize the splitting decision, the bandwidth and computing resource allocation. The optimization problem is multi-objective, and we thus propose a predictive generative adversarial network (GAN)-powered multi-objective optimization algorithm to obtain the Pareto front of the problem. Experimental results show that the proposed algorithm outperforms others in finding Pareto optimal solutions, and the solutions of the proposed HFSL dominate the solution of FL.