Connection Pruning for Deep Spiking Neural Networks with On-Chip Learning

TL;DR

This paper introduces a connection pruning method for deep Spiking Neural Networks that enhances on-chip learning efficiency and energy savings without accuracy loss, suitable for embedded hardware.

Contribution

It presents a novel pruning approach integrated with on-chip STDP learning, significantly reducing network connectivity and improving speed and energy efficiency.

Findings

2.1x speed-up in on-chip learning

64% energy savings during training

92.83% connectivity reduction without accuracy loss

Abstract

Long training time hinders the potential of the deep, large-scale Spiking Neural Network (SNN) with the on-chip learning capability to be realized on the embedded systems hardware. Our work proposes a novel connection pruning approach that can be applied during the on-chip Spike Timing Dependent Plasticity (STDP)-based learning to optimize the learning time and the network connectivity of the deep SNN. We applied our approach to a deep SNN with the Time To First Spike (TTFS) coding and has successfully achieved 2.1x speed-up and 64% energy savings in the on-chip learning and reduced the network connectivity by 92.83%, without incurring any accuracy loss. Moreover, the connectivity reduction results in 2.83x speed-up and 78.24% energy savings in the inference. Evaluation of our proposed approach on the Field Programmable Gate Array (FPGA) platform revealed 0.56% power overhead was needed to implement the pruning algorithm.