The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning

TL;DR

This paper investigates the robustness of Spiking Neural Networks in federated learning and introduces a novel Top-K sparsification algorithm that significantly reduces communication bandwidth while maintaining model accuracy.

Contribution

The paper proposes a new federated learning algorithm with SNNs that reduces communication costs through top-K sparsification and dynamic parameter compression, outperforming existing methods.

Findings

SNNs are inherently robust under noisy communication in FL.

The proposed FLTS algorithm reduces communication to 6% of original model size.

The method maintains high accuracy while significantly decreasing bandwidth usage.

Abstract

Spiking Neural Networks (SNNs) have recently gained significant interest in on-chip learning in embedded devices and emerged as an energy-efficient alternative to conventional Artificial Neural Networks (ANNs). However, to extend SNNs to a Federated Learning (FL) setting involving collaborative model training, the communication between the local devices and the remote server remains the bottleneck, which is often restricted and costly. In this paper, we first explore the inherent robustness of SNNs under noisy communication in FL. Building upon this foundation, we propose a novel Federated Learning with Top-K Sparsification (FLTS) algorithm to reduce the bandwidth usage for FL training. We discover that the proposed scheme with SNNs allows more bandwidth savings compared to ANNs without impacting the model's accuracy. Additionally, the number of parameters to be communicated can be reduced to as low as 6 percent of the size of the original model. We further improve the communication efficiency by enabling dynamic parameter compression during model training. Extensive experiment results demonstrate that our proposed algorithms significantly outperform the baselines in terms of communication cost and model accuracy and are promising for practical network-efficient FL with SNNs.