Sparse-firing regularization methods for spiking neural networks with time-to-first spike coding

TL;DR

This paper introduces two novel regularization methods to reduce firing frequency in time-to-first spike coded spiking neural networks, enhancing energy efficiency while maintaining processing capabilities.

Contribution

The paper proposes membrane potential-aware and firing condition-aware regularization techniques that specifically target sparse firing in TTFS-coded SNNs, requiring only firing timing and weight information.

Findings

Both methods successfully reduce firing frequency in SNNs.

The methods maintain high accuracy on MNIST, Fashion-MNIST, and CIFAR-10.

Energy efficiency of SNNs is improved with minimal performance loss.

Abstract

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs, which is important not only because of its similarity with information processing in the brain, but also from an engineering point of view. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. The first is the membrane potential-aware SSR (M-SSR) method, which has been derived as an extreme form of the loss function of the membrane potential value. The second is the firing condition-aware SSR (F-SSR) method, which is a regularization function obtained from the firing conditions. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.