C3-SL: Circular Convolution-Based Batch-Wise Compression for Communication-Efficient Split Learning

TL;DR

This paper introduces C3-SL, a novel batch-wise compression method for split learning using circular convolution, achieving high compression ratios with minimal accuracy loss and significant reductions in memory and computation overhead.

Contribution

It pioneers batch-wise compression in split learning by leveraging circular convolution and quasi-orthogonality to merge features efficiently.

Findings

Achieves 16x compression ratio with negligible accuracy loss

Reduces memory usage by 1152x compared to existing methods

Decreases computation overhead by 2.25x

Abstract

Most existing studies improve the efficiency of Split learning (SL) by compressing the transmitted features. However, most works focus on dimension-wise compression that transforms high-dimensional features into a low-dimensional space. In this paper, we propose circular convolution-based batch-wise compression for SL (C3-SL) to compress multiple features into one single feature. To avoid information loss while merging multiple features, we exploit the quasi-orthogonality of features in high-dimensional space with circular convolution and superposition. To the best of our knowledge, we are the first to explore the potential of batch-wise compression under the SL scenario. Based on the simulation results on CIFAR-10 and CIFAR-100, our method achieves a 16x compression ratio with negligible accuracy drops compared with the vanilla SL. Moreover, C3-SL significantly reduces 1152x memory and 2.25x computation overhead compared to the state-of-the-art dimension-wise compression method.