Split Learning without Local Weight Sharing to Enhance Client-side Data Privacy

TL;DR

This paper introduces privacy-enhanced split learning (P-SL) without local weight sharing, significantly reducing data leakage, improving privacy, and maintaining accuracy in dynamic multi-client environments.

Contribution

It proposes a novel split learning method without local weight sharing, along with parallelized and cache-based training techniques to enhance privacy, speed, and adaptability.

Findings

P-SL reduces client-side data leakage by up to 50%.

P-SL maintains accuracy comparable to baseline SL across data distributions.

Caching-based training mitigates forgetting and stabilizes learning with late clients.

Abstract

Split learning (SL) aims to protect user data privacy by distributing deep models between client-server and keeping private data locally. In SL training with multiple clients, the local model weights are shared among the clients for local model update. This paper first reveals data privacy leakage exacerbated from local weight sharing among the clients in SL through model inversion attacks. Then, to reduce the data privacy leakage issue, we propose and analyze privacy-enhanced SL (P-SL) (or SL without local weight sharing). We further propose parallelized P-SL to expedite the training process by duplicating multiple server-side model instances without compromising accuracy. Finally, we explore P-SL with late participating clients and devise a server-side cache-based training method to address the forgetting phenomenon in SL when late clients join. Experimental results demonstrate that P-SL helps reduce up to 50% of client-side data leakage, which essentially achieves a better privacy-accuracy trade-off than the current trend by using differential privacy mechanisms. Moreover, P-SL and its cache-based version achieve comparable accuracy to baseline SL under various data distributions, while cost less computation and communication. Additionally, caching-based training in P-SL mitigates the negative effect of forgetting, stabilizes the learning, and enables practical and low-complexity training in a dynamic environment with late-arriving clients.