SL-FAC: A Communication-Efficient Split Learning Framework with Frequency-Aware Compression

TL;DR

SL-FAC introduces a frequency-aware compression framework for split learning, significantly reducing communication overhead while maintaining training effectiveness on resource-limited devices.

Contribution

It proposes a novel adaptive frequency decomposition and quantization method to enhance communication efficiency in split learning.

Findings

SL-FAC reduces communication costs by up to 70%.

The framework maintains comparable model accuracy with less data transmission.

Experiments show improved training speed and efficiency.

Abstract

The growing complexity of neural networks hinders the deployment of distributed machine learning on resource-constrained devices. Split learning (SL) offers a promising solution by partitioning the large model and offloading the primary training workload from edge devices to an edge server. However, the increasing number of participating devices and model complexity leads to significant communication overhead from the transmission of smashed data (e.g., activations and gradients), which constitutes a critical bottleneck for SL. To tackle this challenge, we propose SL-FAC, a communication-efficient SL framework comprising two key components: adaptive frequency decomposition (AFD) and frequency-based quantization compression (FQC). AFD first transforms the smashed data into the frequency domain and decomposes it into spectral components with distinct information. FQC then applies customized quantization bit widths to each component based on its spectral energy distribution. This collaborative approach enables SL-FAC to achieve significant communication reduction while strategically preserving the information most crucial for model convergence. Extensive experiments confirm the superior performance of SL-FAC for improving the training efficiency.