A Time Encoding approach to training Spiking Neural Networks

TL;DR

This paper introduces a novel approach using time encoding theory to interpret and train spiking neural networks, offering an alternative to traditional backpropagation by leveraging spike timing constraints.

Contribution

It applies time encoding concepts to interpret spike times as constraints on weights and develops new training methods for SNNs based on these principles.

Findings

One-layer SNNs can be trained via linear constraints.

Two-layer SNNs training exploits spike timing properties.

Proposes an alternative to backpropagation for SNNs.

Abstract

While Spiking Neural Networks (SNNs) have been gaining in popularity, it seems that the algorithms used to train them are not powerful enough to solve the same tasks as those tackled by classical Artificial Neural Networks (ANNs). In this paper, we provide an extra tool to help us understand and train SNNs by using theory from the field of time encoding. Time encoding machines (TEMs) can be used to model integrate-and-fire neurons and have well-understood reconstruction properties. We will see how one can take inspiration from the field of TEMs to interpret the spike times of SNNs as constraints on the SNNs' weight matrices. More specifically, we study how to train one-layer SNNs by solving a set of linear constraints, and how to train two-layer SNNs by leveraging the all-or-none and asynchronous properties of the spikes emitted by SNNs. These properties of spikes result in an alternative to backpropagation which is not possible in the case of simultaneous and graded activations as in classical ANNs.