Communication-Efficient Split Learning Based on Analog Communication and Over the Air Aggregation

TL;DR

This paper introduces a communication-efficient split learning framework utilizing analog communication and over-the-air aggregation, significantly reducing bandwidth requirements for collaborative inference among many devices.

Contribution

It proposes a novel split learning method that maintains constant communication cost regardless of the number of agents by constraining model weights and biases for over-the-air aggregation.

Findings

Outperforms digital implementations in communication efficiency

Maintains constant communication cost with increasing number of agents

Significantly improves scalability for large multi-agent systems

Abstract

Split-learning (SL) has recently gained popularity due to its inherent privacy-preserving capabilities and ability to enable collaborative inference for devices with limited computational power. Standard SL algorithms assume an ideal underlying digital communication system and ignore the problem of scarce communication bandwidth. However, for a large number of agents, limited bandwidth resources, and time-varying communication channels, the communication bandwidth can become the bottleneck. To address this challenge, in this work, we propose a novel SL framework to solve the remote inference problem that introduces an additional layer at the agent side and constrains the choices of the weights and the biases to ensure over the air aggregation. Hence, the proposed approach maintains constant communication cost with respect to the number of agents enabling remote inference under limited bandwidth. Numerical results show that our proposed algorithm significantly outperforms the digital implementation in terms of communication-efficiency, especially as the number of agents grows large.