LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections

TL;DR

LightSplit introduces a lightweight orthogonal projection in split learning to reduce communication overhead and enhance privacy by limiting information exposure, without sacrificing model accuracy.

Contribution

It proposes a novel orthogonal random projection technique that acts as an information bottleneck, improving privacy and efficiency in split learning without architectural changes.

Findings

Retains over 95% accuracy with 32x dimensionality reduction.

Reduces communication overhead significantly while maintaining training stability.

Effectively limits information exposure against reconstruction attacks.

Abstract

Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server, but the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation, relying on additional mechanisms such as sparsification, quantization, or noise injection. We propose LightSplit, which limits information exposure and reduces communication overhead by applying a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. By transmitting low-dimensional projections instead of raw activations, the server operates on lifted representations without requiring architectural modifications, ensuring compatibility with existing SL architectures. By avoiding additional trainable components on the client, the method remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, LightSplit reduces the information available for reconstruction and limits information exposure. We extensively evaluate LightSplit on state-of-the-art benchmarks in both IID and non-IID settings across varying projection dimensions and client scales. Our results show that the method retains more than 95% of the baseline accuracy at up to 32x reduction in transmitted dimensionality while maintaining stable training dynamics.