GAS: Generative Activation-Aided Asynchronous Split Federated Learning

TL;DR

This paper introduces GAS, a novel asynchronous split federated learning framework that uses generative activations to mitigate bias caused by asynchronous updates, improving model convergence and performance.

Contribution

The paper proposes a new asynchronous SFL framework with generative activations to address bias and delay issues, along with a convergence analysis and experimental validation.

Findings

GAS effectively reduces bias from asynchronous updates.

The method achieves improved convergence bounds.

Experimental results demonstrate enhanced model performance.

Abstract

Split Federated Learning (SFL) splits and collaboratively trains a shared model between clients and server, where clients transmit activations and client-side models to server for updates. Recent SFL studies assume synchronous transmission of activations and client-side models from clients to server. However, due to significant variations in computational and communication capabilities among clients, activations and client-side models arrive at server asynchronously. The delay caused by asynchrony significantly degrades the performance of SFL. To address this issue, we consider an asynchronous SFL framework, where an activation buffer and a model buffer are embedded on the server to manage the asynchronously transmitted activations and client-side models, respectively. Furthermore, as asynchronous activation transmissions cause the buffer to frequently receive activations from resource-rich clients, leading to biased updates of the server-side model, we propose Generative activations-aided Asynchronous SFL (GAS). In GAS, the server maintains an activation distribution for each label based on received activations and generates activations from these distributions according to the degree of bias. These generative activations are then used to assist in updating the server-side model, ensuring more accurate updates. We derive a tighter convergence bound, and our experiments demonstrate the effectiveness of the proposed method. The code is available at https://github.com/eejiarong/GAS.