Vulnerability Due to Training Order in Split Learning

TL;DR

This paper identifies a bias in split learning caused by training order, showing it affects model performance on early clients' data, and proposes SplitFedv3 to mitigate this issue while maintaining privacy.

Contribution

It reveals the training order bias in split learning and introduces SplitFedv3 as a solution to reduce this bias while preserving privacy.

Findings

Model trained on all clients performs poorly on early clients' data.

Bias increases with more clients in the training process.

SplitFedv3 mitigates training order bias effectively.

Abstract

Split learning (SL) is a privacy-preserving distributed deep learning method used to train a collaborative model without the need for sharing of patient's raw data between clients. In split learning, an additional privacy-preserving algorithm called no-peek algorithm can be incorporated, which is robust to adversarial attacks. The privacy benefits offered by split learning make it suitable for practice in the healthcare domain. However, the split learning algorithm is flawed as the collaborative model is trained sequentially, i.e., one client trains after the other. We point out that the model trained using the split learning algorithm gets biased towards the data of the clients used for training towards the end of a round. This makes SL algorithms highly susceptible to the order in which clients are considered for training. We demonstrate that the model trained using the data of all clients does not perform well on the client's data which was considered earliest in a round for training the model. Moreover, we show that this effect becomes more prominent with the increase in the number of clients. We also demonstrate that the SplitFedv3 algorithm mitigates this problem while still leveraging the privacy benefits provided by split learning.