Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks

TL;DR

This paper introduces a fractional-order gradient descent method for training multi-layer spiking neural networks, improving accuracy and efficiency by leveraging fractional calculus in spike-timing-dependent learning.

Contribution

It proposes a novel fractional-order gradient descent approach for SNNs, enhancing training flexibility and performance over traditional methods.

Findings

Accuracy increases with higher fractional order, reaching 155% improvement at order 1.9.

The method achieves state-of-the-art computational efficiency for given network structures.

Demonstrates effective learning on MNIST and DVS128 Gesture datasets.

Abstract

Accumulated detailed knowledge about the neuronal activities in human brains has brought more attention to bio-inspired spiking neural networks (SNNs). In contrast to non-spiking deep neural networks (DNNs), SNNs can encode and transmit spatiotemporal information more efficiently by exploiting biologically realistic and low-power event-driven neuromorphic architectures. However, the supervised learning of SNNs still remains a challenge because the spike-timing-dependent plasticity (STDP) of connected spiking neurons is difficult to implement and interpret in existing backpropagation learning schemes. This paper proposes a fractional-order spike-timing-dependent gradient descent (FO-STDGD) learning model by considering a derived nonlinear activation function that describes the relationship between the quasi-instantaneous firing rate and the temporal membrane potentials of nonleaky integrate-and-fire neurons. The training strategy can be generalized to any fractional orders between 0 and 2 since the FO-STDGD incorporates the fractional gradient descent method into the calculation of spike-timing-dependent loss gradients. The proposed FO-STDGD model is tested on the MNIST and DVS128 Gesture datasets and its accuracy under different network structure and fractional orders is analyzed. It can be found that the classification accuracy increases as the fractional order increases, and specifically, the case of fractional order 1.9 improves by 155% relative to the case of fractional order 1 (traditional gradient descent). In addition, our scheme demonstrates the state-of-the-art computational efficacy for the same SNN structure and training epochs.