KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting

TL;DR

KoReA-SFL introduces a multi-model aggregation and knowledge replay strategy in split federated learning to mitigate data heterogeneity issues and catastrophic forgetting, significantly improving accuracy.

Contribution

It proposes a novel KoReA-SFL framework with multi-model aggregation and knowledge replay to enhance training accuracy in heterogeneous data environments.

Findings

Achieves up to 23.25% test accuracy improvement over traditional SFL.

Effectively mitigates catastrophic forgetting in non-IID data scenarios.

Demonstrates robustness across both IID and non-IID data distributions.

Abstract

Although Split Federated Learning (SFL) is good at enabling knowledge sharing among resource-constrained clients, it suffers from the problem of low training accuracy due to the neglect of data heterogeneity and catastrophic forgetting. To address this issue, we propose a novel SFL approach named KoReA-SFL, which adopts a multi-model aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and a knowledge replay strategy to deal with catastrophic forgetting. Specifically, in KoReA-SFL cloud servers (i.e., fed server and main server) maintain multiple branch model portions rather than a global portion for local training and an aggregated master-model portion for knowledge sharing among branch portions. To avoid catastrophic forgetting, the main server of KoReA-SFL selects multiple assistant devices for knowledge replay according to the training data distribution of each server-side branch-model portion. Experimental results obtained from non-IID and IID scenarios demonstrate that KoReA-SFL significantly outperforms conventional SFL methods (by up to 23.25\% test accuracy improvement).