Federated Split Learning with Improved Communication and Storage Efficiency

TL;DR

This paper introduces CSE-FSL, a federated split learning approach that significantly reduces communication and storage costs by using an auxiliary network and selective data transmission, with proven convergence guarantees.

Contribution

The paper proposes a novel CSE-FSL method that enhances federated split learning by reducing communication and storage overheads through innovative model update strategies and a single server model.

Findings

Achieves substantial communication reduction compared to existing FSL methods.

Maintains convergence guarantees under non-convex loss functions.

Demonstrates effectiveness on real-world federated learning tasks.

Abstract

Federated learning (FL) is one of the popular distributed machine learning (ML) solutions but incurs significant communication and computation costs at edge devices. Federated split learning (FSL) can train sub-models in parallel and reduce the computational burden of edge devices by splitting the model architecture. However, it still requires a high communication overhead due to transmitting the smashed data and gradients between clients and the server in every global round. Furthermore, the server must maintain separate partial models for every client, leading to a significant storage requirement. To address these challenges, this paper proposes a novel communication and storage efficient federated split learning method, termed CSE-FSL, which utilizes an auxiliary network to locally update the weights of the clients while keeping a single model at the server, hence avoiding frequent transmissions of gradients from the server and greatly reducing the storage requirement of the server. Additionally, a new model update method of transmitting the smashed data in selected epochs can reduce the amount of smashed data sent from the clients. We provide a theoretical analysis of CSE-FSL, rigorously guaranteeing its convergence under non-convex loss functions. The extensive experimental results further indicate that CSE-FSL achieves a significant communication reduction over existing FSL solutions using real-world FL tasks.