SLPerf: a Unified Framework for Benchmarking Split Learning

TL;DR

SLPerf is a comprehensive, unified benchmarking framework for split learning that enables fair comparison, evaluation, and development of SL algorithms across diverse scenarios and datasets.

Contribution

This paper introduces SLPerf, the first unified benchmarking library for split learning, along with a detailed survey, extensive experiments, and practical insights for SL research.

Findings

SLPerf enables standardized comparison of SL paradigms.

Extensive experiments reveal performance differences under IID and Non-IID data.

SLPerf facilitates SL algorithm development and evaluation.

Abstract

Data privacy concerns has made centralized training of data, which is scattered across silos, infeasible, leading to the need for collaborative learning frameworks. To address that, two prominent frameworks emerged, i.e., federated learning (FL) and split learning (SL). While FL has established various benchmark frameworks and research libraries,SL currently lacks a unified library despite its diversity in terms of label sharing, model aggregation, and cut layer choice. This lack of standardization makes comparing SL paradigms difficult. To address this, we propose SLPerf, a unified research framework and open research library for SL, and conduct extensive experiments on four widely-used datasets under both IID and Non-IID data settings. Our contributions include a comprehensive survey of recently proposed SL paradigms, a detailed benchmark comparison of different SL paradigms in different situations, and rich engineering take-away messages and research insights for improving SL paradigms. SLPerf can facilitate SL algorithm development and fair performance comparisons. The code is available at https://github.com/Rainysponge/Split-learning-Attacks .